[
{
"path": ".gitignore",
"content": ".idea\n.lein*\ntarget\n.nrepl-port\n*.iml\npom.xml\npom.xml.asc\n/.clj-kondo/\n/.lsp/\n"
},
{
"path": ".tool-versions",
"content": "terraform 1.8.5\nawscli 2.16.10\ntfenv 0.4.0\n"
},
{
"path": ".travis.yml",
"content": "language: clojure\n"
},
{
"path": "CHANGES.md",
"content": "## Changes\n\n_tesla-microservice_ is used for a number of different services now. Still it is a work in progress. This section will document changes and give instructions on breaking ones. Likely you will find corresponding changes in [tesla-examples](https://github.com/otto-de/tesla-examples).\n\n### 0.15.0\nChanged securing of internal endpoints. Moved from providing separate auth-functions\nto components app-status and metering to an auth-middleware provided to the base-system, \nsee [Securing internal info endpoints](https://github.com/otto-de/tesla-microservice#securing-internal-info-endpoints)\n\n### 0.14.0\nThis release cleans up remnants of past eperiments and unused functionality. This leads to breaking changes. \n\n- Remove ```register-timed-handler``` from ```handler``` namespace. Use ```goo/timing-middleware``` instead.\n- Remove ```register-response-fn``` from ```handler``` namespace. This was only used internally.\n- Move internally used middlewares to separate namespaces.\n- Remove support for reporting via graphite from ```metering``` namespace. Only support prometheus reporting via goo and iapetos for the moment.\n- Remove ```SchedulerPool``` protocol from ```scheduler``` namespace. Use ```(:pool scheduler)``` instead of ```(SchedulerPool/pool scheduler)```\n \n\n\n### 0.11.0\nUtilize the iapetos library as main metrics library. Tesla-Microservice is now able to report to graphite as well as prometheus.\nFor configuration of graphite and prometheus reporters please see the updated README.\n\nde.otto.tesla.metrics.prometheus.core now provides some useful instrumentation functions/macros.\n\n### 0.8.0\n\nYou are now able to override the name of the base config file via the runtime config. The following example will make the \nconfiguring component disgregard ```default.edn``` and use ```not-default.edn``` instead. This might be useful when deploying several applications from one repo.\n\n```edn\n{\n :default-cfg-file-name \"not-default\"\n}\n``` \n\n### 0.6.0\n\nThe behaviour of loading configuration changed. \n\n* When using configuration via `properties` files, system properties and environment variables are not loaded by default any more. Use `:merge-env-to-properties-config true` in runtime config to achieve prior behaviour.\n* For the config-file `application.edn`/`application.properties` (name can be overriden by env-var `$CONFIG_FILE`)\n is now with preference loaded as a resource from classpath. If the resource is not found, it is tried to load it as a file.\n\n### 0.5.0 \n\nThe scheduler is now part of the tesla-base-system. \nPer default no threads are kept in the thread-pool it manages.\n\n### 0.4.0\n\nThe scheduler does not have the `de.otto.tesla.stateful.scheduler/schedule` function anymore. \nInstead it only wraps the overtone pool and provides it via `de.otto.tesla.stateful.scheduler/pool`. \nThe pool then can be used with the overtone API like that:\n\n```clj\n(overtone.at-at/every 100 #(println \"Hello world\") (de.otto.tesla.stateful.scheduler/pool scheduler) :desc \"HelloWord Task\")\n```\n\n### 0.1.24\n\nConfig can be provided via EDN-files.\n\nThose files are looked up and merged:\n\n* `default.edn`\n* `{your-custom}.edn`\n* `local.edn`\n\nThe `{your-custom.edn}` can be specified via a ENV-variable named `$CONFIG_FILE`. All\nEDN-config-files have to be located somewhere in the class path.\n\nEven though the old properties-files are considered deprecated and will go away with \nfuture releases, you can still use them, if you specify `:property-file-preferred` in the\nruntime-config of your system:\n\n```edn\n{\n :property-file-preferred true\n}\n``` \n\n### 0.1.17\n\nFix wrapping of middleware to not apply to all routes in the application, which created problems with POST-request.\n\n### 0.1.16\n\nSpeedup of unit-tests (and possibly runtime behaviour) by simpler implmentation of the `:keep-alive`-component.\n\n### 0.1.15\nThe function ```de.otto.tesla.system/start-system``` is renamed to ```start```, ```de.otto.tesla.system/empty-system``` is renamed to ```base-system```. \n\n_tesla-microservice_ does not come with an embedded jetty server out of the box anymore. \n\nTo go on with jetty as before, add the new dependency in ```project.clj```:\n\n```clojure\n [de.otto/tesla-microservice \"0.1.15\"]\n [de.otto/tesla-jetty \"0.1.0\"]\n``` \n\nAdd the server to your system before you start it. Pass any additional dependencies of the server (```:example-page``` in this case).\n\n```clojure\n(system/start (serving-with-jetty/add-server (example-system {}) :example-page))\n```\n\nA working example for this is in the [simple-example](https://github.com/otto-de/tesla-examples/tree/master/simple-example). \nYou can also use the ```->```-threading macro as demonstrated in the [mongo-example](https://github.com/otto-de/tesla-examples/tree/master/mongo-example). \n\n### 0.1.14\nThe `routes`-component was abandoned in favour of the `handler`-component.\nIn the ring library, handlers are the thing to push around (wrapping routes and middleware). You can choose your routing library now. Instead of [compojure](https://github.com/weavejester/compojure) you could also use e.g. [bidi](https://github.com/juxt/bidi).\n\nChange components relying on the old ```routes```-component should be trivial: Instead of adding a vector of (compojure)-routes using ```de.otto.tesla.stateful.routes/register-routes```,\n\n```clojure\n (routes/register-routes\n routes\n [(c/GET \"/test\" [] (test-fn))])\n```\n\njust add a single ring handler using ```de.otto.tesla.stateful.handler/register-handler``` like this:\n\n```clojure\n (handlers/register-handler\n handler\n (c/routes (c/GET \"/test\" [] (test-fn))))\n```\n\nAdd multiple routes like this:\n\n```clojure\n (handlers/register-handler\n handler\n (c/routes (c/GET \"/route1\" [] (test-fn))\n (c/GET \"/route1\" [] (test-fn2))))\n```\n\n\nNote that the keyword for the dependency changed from ```:routes``` to ```:handler``` in the base system.\n\n\n### 0.1.13\nSpecific logging-dependencies and the escaping-messageconverter have been removed. You now have to (read: you are able to) configure logging yourself in your app. To exactly restore the old behaviour add these dependencies to you own application:\n\n```clojure\n[org.slf4j/slf4j-api \"1.7.12\"]\n[ch.qos.logback/logback-core \"1.1.3\"]\n[ch.qos.logback/logback-classic \"1.1.3\"]\n[de.otto/escaping-messageconverter \"0.1.1\"]\n```\n\nin your ```logback.xml``` replace\n```xml\n\n```\n\nwith\n\n```xml\n\n```\n\n\n"
},
{
"path": "Dockerfile",
"content": "# This is an example Dockerfile for running a tesla-microservice app in a docker container.\n#\n# Instructions:\n# 1. build uber jar:\n# ./lein.sh clean\n# ./lein.sh uberjar\n# 2. build docker image\n# docker build -t tesla-example:latest .\n# 3. run docker container\n# docker run -d -p 8080:8080 tesla-example:latest\n\nFROM centos:6\nMAINTAINER Felix Bechstein \nEXPOSE 8080\n\n# prepare image\nRUN yum install -y java-1.8.0-openjdk-headless\nUSER daemon\n\n# set command line\nCMD [\"java\", \"-Dlog_level=info\", \"-jar\", \"/tesla-microservice-standalone.jar\"]\n\n# instead of logging to stdout, you may log to file in /log. create volume or mount host volume to /log\n# RUN mkdir /log && chown daemon /log\n# CMD [\"java\", \"-Dlog_level=info\", \"-Dlog_appender=fileAppender\", \"-Dlog_location=/log\", \"-jar\", \"/tesla-microservice-standalone.jar\"]\n\n# drop in uber jar\nADD target/tesla-microservice-*-standalone.jar /tesla-microservice-standalone.jar\n"
},
{
"path": "LICENSE",
"content": "Apache License\n Version 2.0, January 2004\n http://www.apache.org/licenses/\n\n TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n 1. Definitions.\n\n \"License\" shall mean the terms and conditions for use, reproduction,\n and distribution as defined by Sections 1 through 9 of this document.\n\n \"Licensor\" shall mean the copyright owner or entity authorized by\n the copyright owner that is granting the License.\n\n \"Legal Entity\" shall mean the union of the acting entity and all\n other entities that control, are controlled by, or are under common\n control with that entity. For the purposes of this definition,\n \"control\" means (i) the power, direct or indirect, to cause the\n direction or management of such entity, whether by contract or\n otherwise, or (ii) ownership of fifty percent (50%) or more of the\n outstanding shares, or (iii) beneficial ownership of such entity.\n\n \"You\" (or \"Your\") shall mean an individual or Legal Entity\n exercising permissions granted by this License.\n\n \"Source\" form shall mean the preferred form for making modifications,\n including but not limited to software source code, documentation\n source, and configuration files.\n\n \"Object\" form shall mean any form resulting from mechanical\n transformation or translation of a Source form, including but\n not limited to compiled object code, generated documentation,\n and conversions to other media types.\n\n \"Work\" shall mean the work of authorship, whether in Source or\n Object form, made available under the License, as indicated by a\n copyright notice that is included in or attached to the work\n (an example is provided in the Appendix below).\n\n \"Derivative Works\" shall mean any work, whether in Source or Object\n form, that is based on (or derived from) the Work and for which the\n editorial revisions, annotations, elaborations, or other modifications\n represent, as a whole, an original work of authorship. For the purposes\n of this License, Derivative Works shall not include works that remain\n separable from, or merely link (or bind by name) to the interfaces of,\n the Work and Derivative Works thereof.\n\n \"Contribution\" shall mean any work of authorship, including\n the original version of the Work and any modifications or additions\n to that Work or Derivative Works thereof, that is intentionally\n submitted to Licensor for inclusion in the Work by the copyright owner\n or by an individual or Legal Entity authorized to submit on behalf of\n the copyright owner. For the purposes of this definition, \"submitted\"\n means any form of electronic, verbal, or written communication sent\n to the Licensor or its representatives, including but not limited to\n communication on electronic mailing lists, source code control systems,\n and issue tracking systems that are managed by, or on behalf of, the\n Licensor for the purpose of discussing and improving the Work, but\n excluding communication that is conspicuously marked or otherwise\n designated in writing by the copyright owner as \"Not a Contribution.\"\n\n \"Contributor\" shall mean Licensor and any individual or Legal Entity\n on behalf of whom a Contribution has been received by Licensor and\n subsequently incorporated within the Work.\n\n 2. Grant of Copyright License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n copyright license to reproduce, prepare Derivative Works of,\n publicly display, publicly perform, sublicense, and distribute the\n Work and such Derivative Works in Source or Object form.\n\n 3. Grant of Patent License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n (except as stated in this section) patent license to make, have made,\n use, offer to sell, sell, import, and otherwise transfer the Work,\n where such license applies only to those patent claims licensable\n by such Contributor that are necessarily infringed by their\n Contribution(s) alone or by combination of their Contribution(s)\n with the Work to which such Contribution(s) was submitted. If You\n institute patent litigation against any entity (including a\n cross-claim or counterclaim in a lawsuit) alleging that the Work\n or a Contribution incorporated within the Work constitutes direct\n or contributory patent infringement, then any patent licenses\n granted to You under this License for that Work shall terminate\n as of the date such litigation is filed.\n\n 4. Redistribution. You may reproduce and distribute copies of the\n Work or Derivative Works thereof in any medium, with or without\n modifications, and in Source or Object form, provided that You\n meet the following conditions:\n\n (a) You must give any other recipients of the Work or\n Derivative Works a copy of this License; and\n\n (b) You must cause any modified files to carry prominent notices\n stating that You changed the files; and\n\n (c) You must retain, in the Source form of any Derivative Works\n that You distribute, all copyright, patent, trademark, and\n attribution notices from the Source form of the Work,\n excluding those notices that do not pertain to any part of\n the Derivative Works; and\n\n (d) If the Work includes a \"NOTICE\" text file as part of its\n distribution, then any Derivative Works that You distribute must\n include a readable copy of the attribution notices contained\n within such NOTICE file, excluding those notices that do not\n pertain to any part of the Derivative Works, in at least one\n of the following places: within a NOTICE text file distributed\n as part of the Derivative Works; within the Source form or\n documentation, if provided along with the Derivative Works; or,\n within a display generated by the Derivative Works, if and\n wherever such third-party notices normally appear. The contents\n of the NOTICE file are for informational purposes only and\n do not modify the License. You may add Your own attribution\n notices within Derivative Works that You distribute, alongside\n or as an addendum to the NOTICE text from the Work, provided\n that such additional attribution notices cannot be construed\n as modifying the License.\n\n You may add Your own copyright statement to Your modifications and\n may provide additional or different license terms and conditions\n for use, reproduction, or distribution of Your modifications, or\n for any such Derivative Works as a whole, provided Your use,\n reproduction, and distribution of the Work otherwise complies with\n the conditions stated in this License.\n\n 5. Submission of Contributions. Unless You explicitly state otherwise,\n any Contribution intentionally submitted for inclusion in the Work\n by You to the Licensor shall be under the terms and conditions of\n this License, without any additional terms or conditions.\n Notwithstanding the above, nothing herein shall supersede or modify\n the terms of any separate license agreement you may have executed\n with Licensor regarding such Contributions.\n\n 6. Trademarks. This License does not grant permission to use the trade\n names, trademarks, service marks, or product names of the Licensor,\n except as required for reasonable and customary use in describing the\n origin of the Work and reproducing the content of the NOTICE file.\n\n 7. Disclaimer of Warranty. Unless required by applicable law or\n agreed to in writing, Licensor provides the Work (and each\n Contributor provides its Contributions) on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n implied, including, without limitation, any warranties or conditions\n of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n PARTICULAR PURPOSE. You are solely responsible for determining the\n appropriateness of using or redistributing the Work and assume any\n risks associated with Your exercise of permissions under this License.\n\n 8. Limitation of Liability. In no event and under no legal theory,\n whether in tort (including negligence), contract, or otherwise,\n unless required by applicable law (such as deliberate and grossly\n negligent acts) or agreed to in writing, shall any Contributor be\n liable to You for damages, including any direct, indirect, special,\n incidental, or consequential damages of any character arising as a\n result of this License or out of the use or inability to use the\n Work (including but not limited to damages for loss of goodwill,\n work stoppage, computer failure or malfunction, or any and all\n other commercial damages or losses), even if such Contributor\n has been advised of the possibility of such damages.\n\n 9. Accepting Warranty or Additional Liability. While redistributing\n the Work or Derivative Works thereof, You may choose to offer,\n and charge a fee for, acceptance of support, warranty, indemnity,\n or other liability obligations and/or rights consistent with this\n License. However, in accepting such obligations, You may act only\n on Your own behalf and on Your sole responsibility, not on behalf\n of any other Contributor, and only if You agree to indemnify,\n defend, and hold each Contributor harmless for any liability\n incurred by, or claims asserted against, such Contributor by reason\n of your accepting any such warranty or additional liability.\n\n END OF TERMS AND CONDITIONS\n\n APPENDIX: How to apply the Apache License to your work.\n\n To apply the Apache License to your work, attach the following\n boilerplate notice, with the fields enclosed by brackets \"{}\"\n replaced with your own identifying information. (Don't include\n the brackets!) The text should be enclosed in the appropriate\n comment syntax for the file format. We also recommend that a\n file or class name and description of purpose be included on the\n same \"printed page\" as the copyright notice for easier\n identification within third-party archives.\n\n Copyright {yyyy} {name of copyright owner}\n\n Licensed under the Apache License, Version 2.0 (the \"License\");\n you may not use this file except in compliance with the License.\n You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software\n distributed under the License is distributed on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n See the License for the specific language governing permissions and\n limitations under the License.\n\n"
},
{
"path": "MAINTAINERS",
"content": "Team Tesla \n"
},
{
"path": "OSSMETADATA",
"content": "osslifecycle=active\n"
},
{
"path": "README.md",
"content": "# tesla-microservice\n\n> \"If Edison had a needle to find in a haystack, he would proceed at once with the diligence of the bee to examine straw after straw until he found the object of his search.\" - Nikola Tesla\n\nThis is the common basis for some of otto.de's microservices. It is written in clojure using the [component framework](https://github.com/stuartsierra/component).\n\n[](http://clojars.org/de.otto/tesla-microservice)\n\n[](https://travis-ci.org/otto-de/tesla-microservice)\n[](http://jarkeeper.com/otto-de/tesla-microservice)\n\n\n## Breaking changes\n\n_tesla-microservice_ is used for a number of different services now. Still it is a work in progress. See [CHANGES.md](./CHANGES.md) for instructions on breaking changes.\n\n## Features included\n\n* Load configuration from filesystem.\n* Aggregate a status.\n* Execute functions with a scheduler\n* Reply to a health check.\n* Deliver a json status report.\n* Report to graphite using the metrics library.\n* Manage handlers using ring.\n* Optional auto-hot-reloading of changed source files\n* Shutdown gracefully. If necessary delayed, so load-balancers have time to notice.\n\n## Examples\n\n* A growing set of example applications can be found at [tesla-examples](https://github.com/otto-de/tesla-examples).\n* David & Germán created an example application based, among other, on tesla-microservice. They wrote a very instructive [blog post about it](http://blog.agilityfeat.com/2015/03/clojure-walking-skeleton/)\n* Moritz created [tesla-pubsub-service](https://bitbucket.org/DerGuteMoritz/tesla-pubsub-service). It showcases how to connect components via core.async channels. Also the embedded jetty was replaced by immutant.\n\n### Scheduler\n\nThe scheduler wraps a thread-pool which can be used for scheduling tasks. It is based on [overtones at-at](https://github.com/overtone/at-at) project.\nTo actually use it you have to pass the `:scheduler` as a dependency to the component in which it should be used.\nAfterwards you can schedule tasks using the overtone api like this: \n```clj\n(overtone.at-at/every 100 #(println \"Hello world\") (de.otto.tesla.stateful.scheduler/pool scheduler) :desc \"HelloWord Task\")\n```\n\nThe overtone-pool wrapped by the scheduler can be configured by the config-entry `:scheduler`. (See `overtone.at-at/mk-pool`)\nBy default the pool holds no threads.\n\n### app-status\n\nThe app-status indicates the current status of your microservice. To use it you can register a status function to it.\n\nHere is a simple example for a function that checks if an atom is empty or not.\n\n```clj\n(de.otto.tesla.stateful.app-status/register-status-fun app-status #(status atom))\n``` \n\nThe `app-status` is injected under the keyword :app-status from the base system.\n\n```clj\n(defn status [atom]\n (let [status (if @atom :error :ok)\n message (if @atom \"Atom is empty\" \"Atom is not empty\")]\n (de.otto.status/status-detail :status-id status message)))\n```\n\nFor further information and usages take a look at the: [status library](https://github.com/otto-de/status)\n\n## Choosing a server\n\nAs of version ```0.1.15``` there is no server included any more directly in _tesla-microservice_. \nThis gives you the freedom to a) not use any server at all (e.g. for embedded use) b) choose another server e.g. a non-blocking one like httpkit or immutant. The available options are:\n\n* [tesla-jetty](https://github.com/otto-de/tesla-jetty): The tried and tested embedded jetty.\n* [tesla-httpkit](https://github.com/otto-de/tesla-httpkit): The non-blocking httpkit. \n\n## Configuring\n\nApplications build with `tesla-microservices` can be configured via \n`edn`-files, that have to be located in the class path.\n\nFor backwards compatibility, it is also possible to load config from `properties`-files. \nSee below for noteworthy differences.\n \n\n### Order of loading and merging\n\n1. A file named `default.edn` is loaded as a resource from classpath if present. \n2. A file either named `application.edn` or overridden by the ENV-variable `$CONFIG_FILE`\n is loaded as a resource or, if that is not possible, from the filesystem.\n3. A file name `local.edn` is loaded from classpath if present.\n\nThe configuration hash-map in those files is loaded and merged in the\nspecified order. Which mean configurations for the same key is overridden\nby the latter occurrence.\n\n### ENV-variables\n\nIn contrast to former versions of `tesla-microservice` ENV-variables are not\nmerged into the configuration.\n\nBut you can easily specify ENV-variables, that should be accessible in\nyour configuration:\n\n```edn\n{\n :my-app-secret #ts/env [:my-env-dep-app-secret \"default\"]\n}\n```\n\nENV-variables are read with [environ](https://github.com/weavejester/environ). To see\nwhich keyword represents which ENV-var have a look in their docs. \n\n### Configuring via properties files\n\nFor backwards compatibility, it is also possible to load config from `properties`-files. \nYou'll have to pass `{:property-file-preferred true}` as a runtime config to the base-system.\nIt is not possible to load individual environment variables when using properties config. \nAdding `:merge-env-to-properties-config true` to the runtime config will add all system properties\nand environment variables, overiding any config from files.\n\n### Reporters\nApplications utilizing Tesla-Microservice can use [iapetos prometheus client](https://github.com/xsc/iapetos) for monitoring.\nMetrics are send by reporters which can be configured using the `:metrics` keyword.\nEach configured reporter will start at system startup automatically.\n\nSee example configuration below for all supported reporters.\n\n```clojure\n:metrics {:graphite {:host \"localhost\"\n :port \"2003\"\n :prefix \"my.prefix\"\n :interval-in-s 60\n :include-hostname :first-part}\n :prometheus {:metrics-path \"/metrics\"}}\n```\n\n## Automatic hot-reloading of changed source files\n\nRestarting the whole system after a small change can be cumbersome.\nA _tesla-microservice_ can detect changes to your source files and \nload them into a running server. Add this to your config, to check\nfor changes on each request to your system: \n\n```edn\n{:handler {:hot-reload? true}}\n```\n\n_Note_: This should only be enabled in development mode. \nUse your `local.edn` to enable this feature safely.\nYou can add a `private.edn` as well for personal configurations. This file should be added to your `.gitignore`.\n\n## Securing internal info endpoints\nThe Tesla-Microservice comes with endpoints that hold information about the internal state of your application.\nThose endpoints can be the app-status or even metrics (Prometheus, see above).\nTo secure those endpoints you can provide an authentication-middleware to the base-system. \n\nE.g.:\n\n```clojure\n(defn auth-middleware [config handler-fn]\n (fn [request] (if (authenticated? config request) \n (handler-fn request)\n {:status 401 :body \"access denied\"})))\n\n(defn example-system [runtime-config]\n (-> (de.otto.tesla.system/base-system runtime-config auth-middleware))) \n```\n\n## Addons\n\nThe basis included is stripped to the very minimum. Additional functionality is available as addons:\n\n* [tesla-zookeeper-observer](https://github.com/otto-de/tesla-zookeeper-observer): Read only access to zookeeper.\n* [tesla-mongo-connect](https://github.com/otto-de/tesla-mongo-connect): Read/write access to mongodb.\n* [tesla-cachefile](https://github.com/otto-de/tesla-cachefile): Read and write a cachefile. Locally or in hdfs.\n\nMore features will be released at a later time as separate addons.\n\n## FAQ\n\n**Q:** Is it any good? **A:** Yes.\n\n**Q:** Why tesla? **A:** It's a reference to the ingenious scientist and inventor.\n\n**Q:** Are there alternatives? **A:** Yes. You might want to look at [modularity.org](https://modularity.org/), [system](https://github.com/danielsz/system) and [duct](https://github.com/weavejester/duct).\n\n\n\n## Initial Contributors\n\nChristian Stamm, Felix Bechstein, Ralf Sigmund, Kai Brandes, Florian Weyandt\n\n## License\nReleased under [Apache License 2.0](http://www.apache.org/licenses/LICENSE-2.0) license.\n"
},
{
"path": "dev-resources/logback.xml",
"content": "\n tesla\n\n \n true\n \n %d{ISO8601} %-5p logger=%c thread=%t msg=\"%m\"%n\n \n \n\n \n ${log_location}/de.otto.tesla.log\n true\n\n \n ${log_location}/de.otto.tesla.log.%i\n 1\n 5\n \n\n \n 10MB\n \n\n \n %d{ISO8601} %-5p logger=%c thread=%t msg=\"%m\"%n\n \n \n\n \n \n \n\n\n"
},
{
"path": "project.clj",
"content": "(defproject de.otto/tesla-microservice \"0.17.9-SNAPSHOT\"\n :description \"basic microservice.\"\n :url \"https://github.com/otto-de/tesla-microservice\"\n :license {:name \"Apache License 2.0\"\n :url \"http://www.apache.org/license/LICENSE-2.0.html\"}\n :scm {:name \"git\"\n :url \"https://github.com/otto-de/tesla-microservice\"}\n :repositories [[\"releases\" {:url \"https://repo.clojars.org\"\n :creds :gpg}]]\n :dependencies [[org.clojure/data.json \"2.5.1\"]\n [org.clojure/tools.logging \"1.3.0\"]\n [de.otto/status \"0.1.3\"]\n [de.otto/goo \"1.2.12\"]\n [clojure.java-time \"1.4.3\"]\n [clojurewerkz/propertied \"1.3.0\"]\n [com.stuartsierra/component \"1.1.0\"]\n [compojure \"1.7.1\"]\n [environ \"1.2.0\"]\n [overtone/at-at \"1.4.65\"]\n [ring/ring-core \"1.13.0\"]\n [ring/ring-devel \"1.13.0\"]\n [ring-basic-authentication \"1.2.0\"]]\n\n :exclusions [org.clojure/clojure\n org.slf4j/slf4j-nop\n org.slf4j/slf4j-log4j12\n log4j\n commons-logging/commons-logging]\n :lein-release {:deploy-via :clojars}\n\n :filespecs [{:type :path :path \"test-utils\"}]\n\n :test-selectors {:default (constantly true)\n :integration :integration\n :unit :unit\n :all (constantly true)}\n :profiles {:uberjar {:aot :all}\n :dev {:dependencies [[org.clojure/clojure \"1.12.0\"]\n [org.slf4j/slf4j-api \"2.0.16\"]\n [ch.qos.logback/logback-core \"1.5.14\"]\n [ch.qos.logback/logback-classic \"1.5.14\"]\n [ring-mock \"0.1.5\"]\n [org.clojure/data.codec \"0.2.0\"]]\n :plugins [[lein-release/lein-release \"1.0.9\"]]}}\n :test-paths [\"test\" \"test-resources\" \"test-utils\"])\n"
},
{
"path": "resources/default.edn",
"content": "{\n :status-url \"/status\"\n :health-url \"/health\"\n\n ; metering\n ; take a look at de.otto.tesla.stateful.metering\n ; for all available metrics reporters and their configs\n :metric {:console {:interval-in-s 120}}\n\n :scheduler {:cpu-count 0}\n :server-port \"8080\"\n\n :jetty-options {:send-server-version? false}\n\n ; use this if you need a grace-time during which\n ; the system reports unhealthy before shutting down.\n ;:wait-ms-on-stop \"10000\"\n }\n"
},
{
"path": "resources/default.properties",
"content": "\nstatus.url=/status\nhealth.url=/health\n\n### metering\nmetering.reporter=console\nconsole.interval.seconds=100\n\n### activate graphite like this:\n#metering.reporter=graphite\n#graphite.host=localhost\n#graphite.port=2003\n#graphite.prefix=my-app-prefix\n#graphite.interval.seconds=60\n\nserver.port=8080\n\n## use this if you need a grace-time during which\n## the system reports unhealthy before shutting down.\n#wait.ms.on.stop=10000"
},
{
"path": "src/data_readers.clj",
"content": "{\n ts/env de.otto.tesla.util.env_var_reader/read-env-var\n }\n"
},
{
"path": "src/de/otto/tesla/middleware/exceptions.clj",
"content": "(ns de.otto.tesla.middleware.exceptions\n (:require [clojure.tools.logging :as log]))\n\n(defn exceptions-to-500 [handler]\n (fn [request]\n (try\n (handler request)\n (catch Exception e\n (log/error e \"Will return 500 to client because of this error.\")\n {:status 500\n :body (.getMessage e)}))))"
},
{
"path": "src/de/otto/tesla/stateful/app_status.clj",
"content": "(ns de.otto.tesla.stateful.app-status\n \"This component renders a status page consisting of instance- and configuration-info\n as well as dynamic status of other components\"\n (:require [com.stuartsierra.component :as component]\n [compojure.core :as c]\n [clojure.data.json :as json :only [write-str]]\n [clojure.tools.logging :as log]\n [clojure.string :as str]\n [java-time.api :as jt]\n [environ.core :as env]\n [de.otto.tesla.stateful.handler :as handlers]\n [de.otto.status :as s]\n [ring.middleware.basic-authentication :as ba]\n [de.otto.tesla.util.sanitize :as san]\n [de.otto.tesla.stateful.configuring :as configuring]\n [de.otto.goo.goo :as goo]\n [de.otto.tesla.stateful.auth :as auth]))\n\n(defn keyword-to-status [kw]\n (str/upper-case (name kw)))\n\n(defn status-details-to-json [details]\n (into {} (map\n (fn [[k v]]\n {k (update-in v [:status] keyword-to-status)})\n details)))\n\n(defn system-infos [config]\n {:systemTime (jt/format :iso-date (jt/local-date-time))\n :hostname (configuring/external-hostname config)\n :port (configuring/external-port config)})\n\n(defn aggregation-strategy [config]\n (if (= (get-in config [:status-aggregation]) \"forgiving\")\n s/forgiving-strategy\n s/strict-strategy))\n\n(defn create-complete-status [self]\n (let [config (get-in self [:config :config])\n version-info (get-in self [:config :version])\n aggregate-strategy (:status-aggregation self)\n extra-info {:name (:name config)\n :version (:version version-info)\n :git (:commit version-info)\n :configuration (san/hide-passwds config)}]\n (assoc\n (s/aggregate-status :application\n aggregate-strategy\n @(:status-functions self)\n extra-info)\n :system (system-infos (:config self)))))\n\n(defn status-response-body [self]\n (-> (create-complete-status self)\n (update-in [:application :statusDetails] status-details-to-json)\n (update-in [:application :status] keyword-to-status)))\n\n;; This should apply to the specification at\n;; http://spec.otto.de/media_types/application_vnd_otto_monitoring_status_json.html .\n;; Right now it applies only partially.\n(defn status-response [self _]\n {:status 200\n :headers {\"Content-Type\" \"application/json\"}\n :body (json/write-str (status-response-body self))})\n\n(defn register-status-fun [self fun]\n (swap! (:status-functions self) #(conj % fun)))\n\n(defn path-filter [self handler]\n (let [status-path (get-in self [:config :config :status-url] \"/status\")]\n (c/GET status-path request (handler request))))\n\n(defn mk-handler [{:keys [auth] :as self}]\n ((->> (partial status-response self)\n (goo/timing-middleware)\n (auth/wrap-auth auth)\n (partial path-filter self))))\n\n(defrecord ApplicationStatus [config handler auth]\n component/Lifecycle\n (start [self]\n (log/info \"-> starting Application Status\")\n (let [new-self (assoc self\n :status-aggregation (aggregation-strategy (:config config))\n :status-functions (atom []))]\n\n (handlers/register-handler handler (mk-handler new-self))\n (goo/register-gauge! :build/info {:labels [:version :revision] :description \"Constant '1' value labeled by version and revision of the service.\"})\n (goo/inc! :build/info {:version (-> config :version :version) :revision (-> config :version :commit)})\n new-self))\n\n (stop [self]\n (log/info \"<- stopping Application Status\")\n self))\n\n(defn new-app-status\n ([]\n (map->ApplicationStatus {})))\n"
},
{
"path": "src/de/otto/tesla/stateful/auth.clj",
"content": "(ns de.otto.tesla.stateful.auth\n \"This component handles authentication.\"\n (:require [clojure.tools.logging :as log]\n [com.stuartsierra.component :as component]))\n\n(defn no-auth-middleware [_config handler-fn]\n (log/warn \"You are using no authentication...Is this desired?\")\n (fn [request]\n (handler-fn request)))\n\n(defn wrap-auth [self handler-fn]\n ((:auth-mw self) handler-fn))\n\n(defrecord Auth [config auth-mw]\n component/Lifecycle\n (start [self]\n (log/info \"-> starting AuthMiddleware\")\n (let [auth-mw (partial (or auth-mw no-auth-middleware) (:config config))\n new-self (assoc self :auth-mw auth-mw)]\n new-self))\n (stop [self]\n (log/info \"<- stopping AuthMiddleware\")\n self))\n\n(defn new-auth\n ([auth-mw]\n (map->Auth {:auth-mw auth-mw})))\n"
},
{
"path": "src/de/otto/tesla/stateful/configuring.clj",
"content": "(ns de.otto.tesla.stateful.configuring\n \"This component is responsible for loading the configuration.\"\n (:require [com.stuartsierra.component :as component]\n [clojurewerkz.propertied.properties :as p]\n [clojure.tools.logging :as log]\n [clojure.java.io :as io]\n [de.otto.tesla.util.keyword :as kwutil]\n [environ.core :as env :only [env]]\n [de.otto.tesla.util.env_var_reader :only [read-env-var]]\n [de.otto.tesla.util.sanitize :as san])\n (:import (java.io PushbackReader)))\n\n(defn deep-merge\n \"Recursively merges maps. If vals are not maps, the last value wins.\"\n [& vals]\n (if (every? map? vals)\n (apply merge-with deep-merge vals)\n (last vals)))\n\n(defn- load-properties-from-resource [resource]\n (kwutil/sanitize-keywords\n (p/properties->map\n (p/load-from resource) false)))\n\n(defn resolve-file [name type]\n (cond\n (= :resource type) (io/resource name)\n (and (= :file type) (.exists (io/file name))) (io/file name)\n (and (= :resource-or-file type) (io/resource name)) (io/resource name)\n (and (= :resource-or-file type) (.exists (io/file name))) (io/file name)))\n\n(defn- load-properties [name type]\n (when-let [resource (resolve-file name type)]\n (load-properties-from-resource resource)))\n\n\n(defn- load-edn [name type]\n (when-let [resource (resolve-file name type)]\n (log/debugf \"Reading %s\" name)\n (-> resource\n (io/reader)\n (PushbackReader.)\n (read))))\n\n(defn load-merge [load-fn merge-fn ending runtime-config]\n (let [default-cfg-name (or (:default-cfg-file-name runtime-config) \"default\")\n defaults (load-fn (str default-cfg-name ending) :resource)\n application (load-fn (or (:config-file env/env) (str \"application\" ending)) :resource-or-file)\n local (load-fn (str \"local\" ending) :resource)\n private (load-fn (str \"private\" ending) :resource)\n configs (filter some? [defaults application local private runtime-config])]\n (apply merge-fn configs)))\n\n(defn load-config-from-edn-files [runtime-config]\n (load-merge load-edn deep-merge \".edn\" runtime-config))\n\n(defn load-config-from-properties-files [runtime-config]\n (let [loaded (load-merge load-properties merge \".properties\" runtime-config)]\n (if (:merge-env-to-properties-config runtime-config)\n (merge loaded env/env)\n loaded)))\n\n(defn load-and-merge [runtime-config]\n (if-not (:property-file-preferred runtime-config)\n (load-config-from-edn-files runtime-config)\n (load-config-from-properties-files runtime-config)))\n\n;; Load config on startup.\n(defrecord Configuring [runtime-config]\n component/Lifecycle\n (start [self]\n (log/info \"-> loading configuration.\")\n (log/info runtime-config)\n (let [config (load-and-merge runtime-config)]\n (log/info \"-> using configuration:\\n\" (with-out-str (clojure.pprint/pprint (san/hide-passwds config))))\n (assoc self :config config\n :version (load-properties \"version.properties\" :resource))))\n\n (stop [self]\n (log/info \"<- stopping configuration.\")\n self))\n\n(defn new-config [runtime-config]\n (map->Configuring {:runtime-config runtime-config}))\n\n;; The hostname and port visble from the outside are different for\n;; different environments.\n;; These methods default to Marathon defaults.\n(defn external-hostname [{:keys [config]}]\n (or (:host-name config)\n (:host env/env) (:host-name env/env) (:hostname env/env)\n \"localhost\"))\n\n(defn server-port [config]\n (:server-port config))\n\n;; see above\n(defn external-port [{:keys [config]}]\n (or (server-port config)\n (:port0 env/env) (:host-port env/env) (:server-port env/env)))\n"
},
{
"path": "src/de/otto/tesla/stateful/handler.clj",
"content": "(ns de.otto.tesla.stateful.handler\n \"This component is responsible for collecting HTTP handlers in order to provide them to a server component.\"\n (:require [com.stuartsierra.component :as component]\n [de.otto.tesla.middleware.exceptions :as ex]\n [clojure.tools.logging :as log]\n [ring.middleware.reload :refer [wrap-reload]]))\n\n(defn- single-handler-fn [{:keys [registered-handlers]}]\n (fn [request]\n (some (fn [h] (h request)) @registered-handlers)))\n\n(defn register-handler [{:keys [registered-handlers]} new-handler-fn]\n (swap! registered-handlers conj (ex/exceptions-to-500 new-handler-fn)))\n\n(defn handler [{:keys [config] :as self}]\n (if (get-in config [:config :handler :hot-reload?])\n (wrap-reload (single-handler-fn self))\n (single-handler-fn self)))\n\n(defrecord Handler [config]\n component/Lifecycle\n (start [self]\n (log/info \"-> starting Handler\")\n (assoc self :registered-handlers (atom [])))\n (stop [self]\n (log/info \"<- stopping Handler\")\n self))\n\n(defn new-handler []\n (map->Handler {}))"
},
{
"path": "src/de/otto/tesla/stateful/health.clj",
"content": "(ns de.otto.tesla.stateful.health\n \"This component provides a health-check endpoint which can be used to orchestrate app shutdown with load balancers.\"\n (:require [com.stuartsierra.component :as component]\n [compojure.core :as c]\n [clojure.tools.logging :as log]\n [de.otto.tesla.stateful.handler :as handler]\n [de.otto.goo.goo :as goo]))\n\n(def healthy-response {:status 200\n :headers {\"Content-Type\" \"text/plain\"}\n :body \"HEALTHY\"})\n\n(def unhealthy-response {:status 423\n :headers {\"Content-Type\" \"text/plain\"}\n :body \"UNHEALTHY\"})\n\n(defn health-response [self _]\n (if @(:locked self)\n unhealthy-response\n healthy-response))\n\n(defn path-filter [self handler]\n (let [health-path (get-in self [:config :config :health-url] \"/health\")]\n (c/GET health-path request (handler request))))\n\n(defn make-handler\n [self]\n (->> (partial health-response self)\n goo/timing-middleware\n (path-filter self)))\n\n(defn lock-application [self]\n (goo/update! :health/locked 0)\n (reset! (:locked self) true))\n\n(defrecord Health [config handler]\n component/Lifecycle\n (start [self]\n (log/info \"-> Starting healthcheck.\")\n (let [new-self (assoc self :locked (atom false))]\n (handler/register-handler handler (make-handler new-self)) ;; TODO: use config directly\n (goo/register-gauge! :health/locked {})\n (goo/update! :health/locked 1)\n new-self))\n\n (stop [self]\n (log/info \"<- Stopping Healthcheck\")\n self))\n\n(defn new-health []\n (map->Health {}))\n\n"
},
{
"path": "src/de/otto/tesla/stateful/keep_alive.clj",
"content": "(ns de.otto.tesla.stateful.keep-alive\n \"This component is responsible for keeping the system alive by creating a non-deamonized noop thread.\"\n (:require [clojure.tools.logging :as log]\n [com.stuartsierra.component :as component])\n (:import (java.util.concurrent CountDownLatch)))\n\n(defn exit-keep-alive []\n (log/info \"<- stopping keepalive thread: \" (.getName (Thread/currentThread))))\n\n(defn enter-keep-alive []\n (log/info \"-> starting keepalive thread: \" (.getName (Thread/currentThread))))\n\n(defn wait-for-count-down-latch [cdl]\n (enter-keep-alive)\n (.await cdl)\n (exit-keep-alive))\n\n(defn start-keep-alive-thread [cd-latch]\n (doto (Thread. ^Runnable (partial wait-for-count-down-latch cd-latch) \"tesla-ms-keep-alive\")\n (.start)))\n\n(defrecord KeepAlive [cd-latch]\n component/Lifecycle\n (start [self]\n (log/info \"-> starting keepalive\")\n (let [cd-latch (CountDownLatch. 1)]\n (assoc self\n :thread (start-keep-alive-thread cd-latch)\n :cd-latch cd-latch)))\n\n (stop [self]\n (log/info \"<- stopping keepalive\")\n (.countDown cd-latch)\n (dissoc self :thread)))\n\n(defn new-keep-alive []\n (map->KeepAlive {}))\n"
},
{
"path": "src/de/otto/tesla/stateful/metering.clj",
"content": "(ns de.otto.tesla.stateful.metering\n \"This component handles exporting of (prometheus) metrics\"\n (:require\n [com.stuartsierra.component :as component]\n [clojure.tools.logging :as log]\n [de.otto.goo.goo :as goo]\n [de.otto.tesla.stateful.handler :as handlers]\n [compojure.core :as c]\n [overtone.at-at :as at]\n [de.otto.tesla.stateful.auth :as auth]))\n\n(defn write-to-console []\n (log/info \"Metrics Reporting:\\n\" (goo/text-format)))\n\n(defn start-console-reporter [console-config scheduler]\n (let [interval-in-ms (* 1000 (:interval-in-s console-config))]\n (log/info \"Starting metrics console reporter\")\n (at/every interval-in-ms write-to-console (:pool scheduler) :desc \"Console-Reporter\")))\n\n(defn metrics-response [_]\n (fn [_request] {:status 200\n :headers {\"Content-Type\" \"text/plain\"}\n :body (goo/text-format)}))\n\n(defn- path-filter [metrics-path handler]\n (c/GET metrics-path request (handler request)))\n\n(defn register-metrics-endpoint [{metrics-path :metrics-path} {:keys [handler auth]}]\n (log/info \"Register metrics prometheus endpoint\")\n (handlers/register-handler handler ((->> (metrics-response handler)\n (goo/timing-middleware)\n (auth/wrap-auth auth)\n (partial path-filter metrics-path)))))\n\n(defn- start-reporter! [{:keys [scheduler] :as self} [reporter-type reporter-config]]\n (case reporter-type\n :console (start-console-reporter reporter-config scheduler)\n :prometheus (register-metrics-endpoint reporter-config self)))\n\n(defn- start-reporters! [{:keys [config] :as self}]\n (let [available-reporters (get-in config [:config :metrics])]\n (run! (partial start-reporter! self) available-reporters)))\n\n(defrecord Metering [config handler scheduler auth]\n component/Lifecycle\n (start [self]\n (log/info \"-> starting metering.\")\n (goo/register-counter! :metering/errors {:labels [:error :metric-name]})\n (assoc self :reporters (start-reporters! self)))\n\n (stop [self]\n (log/info \"<- stopping metering\")\n self))\n\n(defn new-metering\n ([]\n (map->Metering {})))\n"
},
{
"path": "src/de/otto/tesla/stateful/scheduler.clj",
"content": "(ns de.otto.tesla.stateful.scheduler\n \"This components maintains a thread pool which can be used to schedule activities.\"\n (:require [com.stuartsierra.component :as c]\n [clojure.tools.logging :as log]\n [overtone.at-at :as ot]\n [de.otto.tesla.stateful.app-status :as app-status])\n (:import (java.util.concurrent ScheduledThreadPoolExecutor)))\n\n(defn- as-seq [v]\n (apply concat v))\n\n(defn- new-ot-pool [config]\n (let [pool-config (get-in config [:config :scheduler])]\n (apply ot/mk-pool (as-seq pool-config))))\n\n(defn- as-readable-time [l]\n (.format (java.text.SimpleDateFormat. \"EEEE HH':'mm':'ss's'\") l))\n\n(defn- job-details [{:keys [id created-at initial-delay desc ms-period scheduled?]}]\n [(keyword (str id)) {:desc desc\n :createdAt (as-readable-time created-at)\n :initialDelay initial-delay\n :msPeriod ms-period\n :scheduled? @scheduled?}])\n\n(defn- pool-details [{:keys [pool-atom]}]\n (when pool-atom\n (let [^ScheduledThreadPoolExecutor thread-pool (:thread-pool @pool-atom)]\n {:active (.getActiveCount thread-pool)\n :queueSize (.size (.getQueue thread-pool))\n :poolSize (.getPoolSize thread-pool)})))\n\n(defn- scheduler-app-status [{:keys [pool]}]\n {:scheduler {:status :ok\n :poolInfo (pool-details pool)\n :scheduledJobs (into {} (map job-details (ot/scheduled-jobs pool)))}})\n\n(defn exception-to-log [desc f]\n (try (f)\n (catch Exception e\n (log/error e (str \"Exception during scheduled job: \" desc)))))\n\n(defn every\n \"Calls fun every ms-period, and takes an optional initial-delay for\n the first call in ms. Returns a scheduled-fn which may be cancelled\n with cancel. All exceptions are catched and logged.\n\n Default options are\n {:initial-delay 0 :desc \\\"\\\"}\"\n\n [{:keys [pool]} ms-period fun & {:keys [initial-delay desc]\n :or {initial-delay 0\n desc \"\"}}]\n (ot/every ms-period (partial exception-to-log desc fun) pool :initial-delay initial-delay :desc desc))\n\n(defn interspaced\n \"Calls fun repeatedly with an interspacing of ms-period, i.e. the next\n call of fun will happen ms-period milliseconds after the completion\n of the previous call. Also takes an optional initial-delay for the\n first call in ms. Returns a scheduled-fn which may be cancelled with\n cancel. All exceptions are catched and logged.\n\n Default options are\n {:initial-delay 0 :desc \\\"\\\"}\"\n [{:keys [pool]} ms-period fun & {:keys [initial-delay desc]\n :or {initial-delay 0\n desc \"\"}}]\n (ot/interspaced ms-period (partial exception-to-log desc fun) pool :initial-delay initial-delay :desc desc))\n\n\n(defrecord Scheduler [config app-status]\n c/Lifecycle\n (start [self]\n (log/info \"-> Start Scheduler\")\n (let [new-self (assoc self :pool (new-ot-pool config))]\n (app-status/register-status-fun app-status (partial scheduler-app-status new-self))\n new-self))\n\n (stop [self]\n (log/info \"<- Stop Scheduler\")\n (when-let [pool (:pool self)]\n (ot/stop-and-reset-pool! pool))\n self))\n\n(defn new-scheduler []\n (map->Scheduler {}))\n"
},
{
"path": "src/de/otto/tesla/system.clj",
"content": "(ns de.otto.tesla.system\n (:require [com.stuartsierra.component :as c]\n [de.otto.tesla.stateful.app-status :as app-status]\n [de.otto.tesla.stateful.health :as health]\n [de.otto.goo.goo :as goo]\n [de.otto.tesla.stateful.configuring :as configuring]\n [de.otto.tesla.stateful.metering :as metering]\n [de.otto.tesla.stateful.keep-alive :as keep-alive]\n [de.otto.tesla.stateful.scheduler :as scheduler]\n [clojure.tools.logging :as log]\n [environ.core :as env :only [env]]\n [de.otto.tesla.stateful.handler :as handler]\n [de.otto.tesla.stateful.auth :as auth])\n (:import (clojure.lang ExceptionInfo)))\n\n(defn wait! [system]\n (when-let [wait-time (get-in system [:config :config :wait-ms-on-stop])]\n (try\n (log/info \"<- Waiting \" wait-time \" milliseconds.\")\n (Thread/sleep (Long/parseLong wait-time))\n (catch Exception e (log/error e)))))\n\n(defn- exit [code]\n (System/exit code))\n\n(defn- try-stop [system]\n (try\n (c/stop system)\n (log/info \"System stopped. Bye.\")\n (catch Exception ex\n (log/error ex \"Error on stopping the system.\")\n (exit 1))))\n\n(defn stop [system]\n (when-let [sdt (:sdt system)]\n (.removeShutdownHook (Runtime/getRuntime) sdt))\n (when-let [health (:health system)]\n (log/info \"<- System will be stopped. Setting lock.\")\n (health/lock-application health)\n (wait! system))\n (log/info \"<- Stopping system.\")\n (try-stop system))\n\n(defn- try-start [system]\n (try\n (c/start system)\n (catch ExceptionInfo e\n (log/error (c/ex-without-components e) \"Going to shut down because of this error.\")\n (-> e (ex-data) :system (try-stop)))))\n\n(defn start [system]\n (log/info \"-> Starting system.\")\n (let [start-timestamp (System/currentTimeMillis)\n started (try-start system)]\n (log/info \"-> System completely started.\")\n (goo/register-counter! :system-startups {:description \"Counts startups.\"})\n (goo/register-counter! :system-startup-duration-seconds {:description \"Measures the startup duration of the system.\"})\n (goo/inc! :system-startups)\n (goo/inc! :system-startup-duration-seconds (/ (- (System/currentTimeMillis) start-timestamp) 1000))\n (if (map? started)\n (let [sdt (Thread. ^Runnable (partial stop started))]\n (.addShutdownHook (Runtime/getRuntime) sdt)\n (assoc started :sdt sdt))\n started)))\n\n(map? [])\n\n(defn base-system [runtime-config & [auth-mw]]\n (c/system-map\n :keep-alive (keep-alive/new-keep-alive)\n :config (c/using (configuring/new-config runtime-config) [:keep-alive])\n :handler (c/using (handler/new-handler) [:config])\n :metering (c/using (metering/new-metering) [:config :handler :scheduler :auth])\n :health (c/using (health/new-health) [:config :handler])\n :app-status (c/using (app-status/new-app-status) [:config :handler :auth])\n :scheduler (c/using (scheduler/new-scheduler) [:config :app-status])\n :auth (c/using (auth/new-auth auth-mw) [:config])))\n"
},
{
"path": "src/de/otto/tesla/util/env_var_reader.clj",
"content": "(ns de.otto.tesla.util.env_var_reader\n (:require [environ.core :as env]))\n\n(defn read-env-var\n ([[env-var-key fallback]]\n (or\n (get env/env env-var-key fallback)\n \"\")))\n"
},
{
"path": "src/de/otto/tesla/util/keyword.clj",
"content": "(ns de.otto.tesla.util.keyword\n (:require [clojure.string :as str]))\n\n;implementation is copied from environ 1.0.0\n(defn- keywordize [s]\n (-> (str/lower-case s)\n (str/replace \"_\" \"-\")\n (str/replace \".\" \"-\")\n (keyword)))\n\n(defn sanitize-keywords [m]\n (->> m\n (map (fn [[k v]] [(keywordize k) v]))\n (into {})))\n\n;; removed keywordize-keys, provided by clojure.walk/keywordize-keys\n"
},
{
"path": "src/de/otto/tesla/util/sanitize.clj",
"content": "(ns de.otto.tesla.util.sanitize)\n\n(def checklist [\"password\" \"pw\" \"passwd\" \"private\" \"secret\" \"token\"])\n\n(defn hide-passwd [k v]\n (if (some true? (map #(.contains (name k) %) checklist))\n \"***\"\n v))\n\n(defn hide-passwds [map]\n (reduce\n (fn [new-map [k v]] (assoc new-map k (if (map? v) (hide-passwds v) (hide-passwd k v))))\n {} map))\n"
},
{
"path": "test/de/otto/tesla/reporter/prometheus_test.clj",
"content": "(ns de.otto.tesla.reporter.prometheus-test\n (:require [clojure.test :refer :all]\n [de.otto.tesla.system :as system]\n [com.stuartsierra.component :as c]\n [clojure.data.codec.base64 :as b64]\n [de.otto.tesla.stateful.metering :as metering]\n [de.otto.tesla.stateful.handler :as handler]\n [ring.mock.request :as mock]\n [ring.middleware.basic-authentication :as ba]))\n\n(defn- to-base64 [original]\n (String. ^bytes (b64/encode (.getBytes original)) \"UTF-8\"))\n\n(defn- auth-header [request user password]\n (mock/header request \"authorization\" (str \"Basic \" (to-base64 (str user \":\" password)))))\n\n(defn system [runtime-config auth-middleware]\n (-> (system/base-system runtime-config auth-middleware)\n (dissoc :server)))\n\n(defn- handlers [runtime-config & [auth-middleware]]\n (let [system (system runtime-config auth-middleware)\n started-system (c/start-system system)]\n (handler/handler (:handler started-system))))\n\n(defn- rc-metrics-request [system-handler user password]\n (-> (mock/request :get \"/metrics\")\n (auth-header user password)\n (system-handler)\n :status))\n\n(deftest authentication\n (let [config {:metrics {:prometheus {:metrics-path \"/metrics\"}}}\n auth-fun (fn [usr pw]\n (and\n (= \"some-user\" usr) (= \"some-password\" pw)))\n auth-middleware (fn [_config handler]\n (ba/wrap-basic-authentication handler auth-fun))\n system-handler (handlers config auth-middleware)]\n (testing \"it should allow access if authentication succeeds\"\n (is (= 200 (rc-metrics-request system-handler \"some-user\" \"some-password\"))))\n (testing \"it should deny access if authentication fails\"\n (is (= 401 (rc-metrics-request system-handler \"some-user\" \"wrong password\"))))))\n\n\n"
},
{
"path": "test/de/otto/tesla/stateful/app_status_test.clj",
"content": "(ns de.otto.tesla.stateful.app-status-test\n (:require [clojure.test :refer :all]\n [de.otto.tesla.stateful.app-status :as app-status]\n [com.stuartsierra.component :as c]\n [environ.core :as env]\n [clojure.data.json :as json]\n [clojure.tools.logging :as log]\n [de.otto.tesla.util.test-utils :as u]\n [de.otto.tesla.system :as system]\n [de.otto.tesla.stateful.handler :as handler]\n [ring.mock.request :as mock]\n [de.otto.status :as s]\n [de.otto.goo.goo :as goo]\n [ring.middleware.basic-authentication :as ba]))\n\n(defn- serverless-system [runtime-config & [auth-middleware]]\n (dissoc\n (system/base-system runtime-config auth-middleware)\n :server))\n\n(deftest ^:unit should-have-system-status-for-runtime-config\n (u/with-started [system (serverless-system {:host-name \"bar\" :server-port \"0123\"})]\n (let [status (:app-status system)\n system-status (:system (app-status/status-response-body status))]\n (is (= (:hostname system-status) \"bar\"))\n (is (= (:port system-status) \"0123\"))\n (is (not (nil? (:systemTime system-status)))))))\n\n(deftest ^:unit host-name-and-port-on-app-status\n (with-redefs [env/env {:host-name \"foo\" :server-port \"1234\"}]\n (testing \"should add host and port from env to app-status in property-file case\"\n (u/with-started [system (serverless-system {:property-file-preferred true :merge-env-to-properties-config true})]\n (let [status (:app-status system)\n system-status (:system (app-status/status-response-body status))]\n (is (= (:hostname system-status) \"foo\"))\n (is (= (:port system-status) \"1234\"))\n (is (not (nil? (:systemTime system-status)))))))\n (testing \"should add host and port from env to app-status in edn-file case\"\n (u/with-started [system (serverless-system {} nil)]\n (let [status (:app-status system)\n system-status (:system (app-status/status-response-body status))]\n (is (= (:hostname system-status) \"foo\"))\n (is (= (:port system-status) \"9991\"))\n (is (not (nil? (:systemTime system-status)))))))))\n\n(defrecord MockStatusSource [response]\n c/Lifecycle\n (start [self]\n (app-status/register-status-fun (:app-status self) #(:response self))\n self)\n (stop [self]\n self))\n\n(defn- mock-status-system [response]\n (assoc (serverless-system {})\n :mock-status\n (c/using (map->MockStatusSource {:response response}) [:app-status])))\n\n(deftest ^:unit should-show-applicationstatus\n (u/with-started [started (mock-status-system {:mock {:status :ok\n :message \"nevermind\"}})]\n (let [status (:app-status started)\n page (app-status/status-response status {})\n _ (log/info page)\n application-body (get (json/read-str (:body page)) \"application\")]\n (testing \"it shows OK as application status\"\n (is (= (get application-body \"status\")\n \"OK\")))\n\n (testing \"it shows the substatus\"\n (is (= (get application-body \"statusDetails\")\n {\"mock\" {\"message\" \"nevermind\"\n \"status\" \"OK\"}\n \"scheduler\" {\"poolInfo\" {\"active\" 0\n \"poolSize\" 0\n \"queueSize\" 0}\n \"scheduledJobs\" {}\n \"status\" \"OK\"}}))))))\n\n(deftest ^:unit should-show-warning-as-application-status\n (u/with-started [started (mock-status-system {:mock {:status :warning\n :message \"nevermind\"}})]\n (let [status (:app-status started)\n page (app-status/status-response status {})\n applicationStatus (get (get (json/read-str (:body page)) \"application\") \"status\")]\n (is (= applicationStatus \"WARNING\")))))\n\n\n(deftest ^:integration should-serve-status-under-configured-url\n (testing \"use the default url\"\n (u/with-started [started (serverless-system {})]\n (let [handlers (handler/handler (:handler started))]\n (is (= (:status (handlers (mock/request :get \"/status\")))\n 200)))))\n\n (testing \"use the configuration url\"\n (u/with-started [started (serverless-system {:status-url \"/my-status\"})]\n (let [handlers (handler/handler (:handler started))]\n (is (= (:status (handlers (mock/request :get \"/my-status\")))\n 200)))))\n\n (testing \"default should be overridden\"\n (u/with-started [started (serverless-system {:status-url \"/my-status\"})]\n (let [handlers (handler/handler (:handler started))]\n (is (= (handlers (mock/request :get \"/status\"))\n nil)))))\n (testing \"response should be metered\"\n (goo/clear-default-registry!)\n (u/with-started [started (serverless-system {})]\n (let [handlers (handler/handler (:handler started))]\n (handlers (mock/request :get \"/status\"))\n (u/eventually (= 1.0\n (last (.-buckets (.get ((goo/snapshot) :http/duration_in_s {:path \"/status\" :method :get :rc 200}))))))))))\n\n(deftest should-add-version-properties-to-status\n (testing \"it should add the version properties\"\n (u/with-started [started (serverless-system {})]\n (let [handlers (handler/handler (:handler started))\n request (mock/request :get \"/status\")\n status-map (json/read-json (:body (handlers request)))]\n (is (= (get-in status-map [:application :version]) \"test.version\"))\n (is (= (get-in status-map [:application :git]) \"test.githash\"))))))\n\n(deftest determine-status-strategy\n (testing \"it should use strict stategy if none is configured\"\n (let [config {:status-aggregation nil}]\n (is (= (app-status/aggregation-strategy config) s/strict-strategy))))\n\n (testing \"it should use forgiving stategy if forgiving is configured\"\n (let [config {:status-aggregation \"forgiving\"}]\n (is (= (app-status/aggregation-strategy config) s/forgiving-strategy))))\n\n (testing \"it should use strict stategy if something else is configured\"\n (let [config {:status-aggregation \"unknown\"}]\n (is (= (app-status/aggregation-strategy config) s/strict-strategy)))))\n\n(defn start-authenticated-system [user password]\n (let [config {}\n auth-fn (fn [usr pw] (and (= user usr) (= password pw)))\n auth-middleware (fn [_config handler] (ba/wrap-basic-authentication handler auth-fn))\n system (serverless-system config auth-middleware)\n started-system (c/start-system system)]\n (handler/handler (:handler started-system))))\n\n(deftest authentication\n (let [handlers (start-authenticated-system \"some-user\" \"some-password\")]\n (testing \"it should allow access if authentication succeeds\"\n (is (= 200 (:status (handlers (mock/header (mock/request :get \"/status\") \"authorization\" \"Basic c29tZS11c2VyOnNvbWUtcGFzc3dvcmQ=\"))))))\n (testing \"it should deny access if authentication fails\"\n (is (= 401 (:status (handlers (mock/header (mock/request :get \"/status\") \"authorization\" \"Basic c29tZS11c2VyOnNvbWUtcGEzc3dvcmQ=\"))))))))"
},
{
"path": "test/de/otto/tesla/stateful/configuring_test.clj",
"content": "(ns de.otto.tesla.stateful.configuring-test\n (:require [clojure.test :refer :all]\n [de.otto.tesla.stateful.configuring :as configuring]\n [com.stuartsierra.component :as component]\n [clojure.java.io :as io]\n [de.otto.tesla.util.test-utils :as u]\n [environ.core :as env]))\n\n(defn- test-system [rt-conf]\n (-> (component/system-map\n :conf (configuring/new-config rt-conf))))\n\n(deftest referencing-env-properties\n (testing \"should return env-property if referenced in edn-config\"\n (with-redefs [env/env {:prop-without-fallback \"prop-value\"}]\n (u/with-started [started (test-system {})\n conf (get-in started [:conf :config])]\n (is (= (:prop-without-fallback conf) \"prop-value\")))))\n (testing \"should return empty if env prop does not exist and fallback not provided\"\n (with-redefs [env/env {}]\n (u/with-started [started (test-system {})\n conf (get-in started [:conf :config])]\n (is (= (:prop-without-fallback conf) \"\"))))))\n\n(deftest ^:unit should-read-property-from-default-config\n (testing \"should be possible to prefer reading configs from property files\"\n (u/with-started [started (test-system {:property-file-preferred true})]\n (let [conf (get-in started [:conf :config])]\n (is (= (:foo-prop conf) \"baz\"))\n (is (= (get-in conf [:foo :edn]) nil))))))\n\n(deftest ^:unit should-read-property-from-default-edn-file\n (u/with-started [started (test-system {})]\n (let [edn-conf (get-in started [:conf :config])]\n (is (= (:foo-prop edn-conf) nil))\n (is (= (get-in edn-conf [:foo :edn]) \"baz\")))))\n\n(deftest ^:unit should-read-property-from-private-edn-file\n (u/with-started [started (test-system {})]\n (let [conf (get-in started [:conf :config])]\n (is (= (:very conf) :private)))))\n\n(deftest ^:unit should-read-property-from-custom-edn-file\n (with-redefs [env/env {:config-file \"./test-resources/test.edn\"}]\n (u/with-started [started (test-system {})]\n (let [edn-conf (get-in started [:conf :config])]\n (is (= (get-in edn-conf [:health-url]) \"/test/health\"))\n (is (= (get-in edn-conf [:foo :local]) true))\n (is (= (get-in edn-conf [:foo :edn]) \"baz\"))))))\n\n(deftest ^:unit should-ignore-missing-custom-edn-file\n (with-redefs [env/env {:config-file \"non-existing.edn\"}]\n (u/with-started [started (test-system {:runtime 123})]\n (let [edn-conf (get-in started [:conf :config])]\n (is (= (get-in edn-conf [:runtime]) 123))\n (is (= (get-in edn-conf [:health-url]) \"/health\"))\n (is (= (get-in edn-conf [:foo :edn]) \"baz\"))))))\n\n(deftest ^:unit should-read-property-from-runtime-config\n (u/with-started [started (test-system {:foo-rt \"bat\" :foo {:nested 123}})]\n (let [edn-conf (get-in started [:conf :config])]\n (is (= (:foo-prop edn-conf) nil))\n (is (= (:foo-rt edn-conf) \"bat\"))\n (is (= (get-in edn-conf [:foo :edn]) \"baz\"))\n (is (= (get-in edn-conf [:foo :nested]) 123)))))\n\n(deftest ^:unit should-read-default-properties\n (testing \"should read default properties from property-files\"\n (let [loaded-properties (configuring/load-config-from-properties-files {})]\n (is (not (nil? (:server-port loaded-properties))))\n (is (not (nil? (:metering-reporter loaded-properties))))))\n\n (testing \"should read default properties from edn-property-files\"\n (let [loaded-properties (configuring/load-config-from-edn-files {})]\n (is (= \"9991\"\n (:server-port loaded-properties)))\n (is (nil? (:metering-reporter loaded-properties))))))\n\n(deftest ^:unit determine-hostname-from-config-and-env-with-defined-precedence\n (testing \"it prefers a explicitly configured :host-name\"\n (with-redefs [env/env {:host \"host\" :host-name \"host-name\" :hostname \"hostname\"}]\n (u/with-started [started (test-system {:host-name \"configured\"})]\n (is (= \"configured\"\n (configuring/external-hostname (:conf started)))))))\n (testing \"it falls back to env-vars and prefers $HOST\"\n (with-redefs [env/env {:host \"host\" :host-name \"host-name\" :hostname \"hostname\"}]\n (u/with-started [started (test-system {})]\n (is (= \"host\"\n (configuring/external-hostname (:conf started)))))))\n (testing \"it falls back to env-vars and prefers $HOST_NAME\"\n (with-redefs [env/env {:host-name \"host-name\" :hostname \"hostname\"}]\n (u/with-started [started (test-system {})]\n (is (= \"host-name\"\n (configuring/external-hostname (:conf started)))))))\n (testing \"it falls back to env-vars and looks finally for $HOSTNAME\"\n (with-redefs [env/env {:hostname \"hostname\"}]\n (u/with-started [started (test-system {})]\n (is (= \"hostname\"\n (configuring/external-hostname (:conf started)))))))\n (testing \"it eventually falls back to localhost\"\n (with-redefs [env/env {:host-name nil :hostname nil :host nil}]\n (u/with-started [started (test-system {})]\n (is (= \"localhost\"\n (configuring/external-hostname (:conf started))))))))\n\n(deftest ^:unit determine-hostport-from-config-and-env-with-defined-precedence\n (testing \"it prefers a explicitly configured :hostname\"\n (with-redefs [configuring/server-port (constantly \"configured\")]\n (with-redefs [env/env {:port0 \"0\" :host-port \"1\" :server-port \"2\"}]\n (is (= \"configured\"\n (configuring/external-port {}))))))\n\n (with-redefs [configuring/server-port (constantly nil)]\n (testing \"it falls back to env-vars and prefers $PORT0\"\n (with-redefs [env/env {:port0 \"0\" :host-port \"1\" :server-port \"2\"}]\n (u/with-started [started (test-system {})]\n (is (= \"0\"\n (configuring/external-port {}))))))\n (testing \"it falls back to env-vars and prefers $HOST_PORT\"\n (with-redefs [env/env {:host-port \"1\" :server-port \"2\"}]\n (u/with-started [started (test-system {})]\n (is (= \"1\"\n (configuring/external-port {})))))))\n (testing \"it falls back to env-vars and finally takes $SERVER_PORT\"\n (with-redefs [env/env {:server-port \"2\"}]\n (u/with-started [started (test-system {})]\n (is (= \"2\"\n (configuring/external-port {})))))))\n\n(deftest ^:integration should-read-properties-from-file\n (spit \"application.properties\" \"foooo=barrrr\")\n (is (= (:foooo (configuring/load-config-from-properties-files {}))\n \"barrrr\"))\n (io/delete-file \"application.properties\"))\n\n(deftest ^:integration should-prefer-configured-conf-file\n (spit \"application.properties\" \"foooo=value\")\n (spit \"other.properties\" \"foooo=other-value\")\n (with-redefs-fn {#'env/env {:config-file \"other.properties\"}}\n #(is (= (:foooo (configuring/load-config-from-properties-files {}))\n \"other-value\")))\n (io/delete-file \"other.properties\")\n (io/delete-file \"application.properties\"))\n\n(deftest ^:unit deep-merge-test\n (testing \"simple cases\"\n (is (= {:a 1 :b 2}\n (configuring/deep-merge {:a 1}\n {:b 2})))\n (is (= {:a 1 :b 2}\n (configuring/deep-merge {:a 1 :b 1}\n {:b 2})))\n (is (= {:a 2 :b 2}\n (configuring/deep-merge {:a 1 :b 1}\n {:a 2 :b 2})))\n (is (= {:a 3 :b 2 :c 3}\n (configuring/deep-merge {:a 1 :b 1}\n {:b 2}\n {:a 3 :c 3}))))\n (testing \"nested maps\"\n (is (= {:a {:b {:c 1 :d 2}}}\n (configuring/deep-merge {:a {:b {:c 1}}}\n {:a {:b {:d 2}}})))\n (is (= {:a {:b {:c 2 :d 2} :f 3}}\n (configuring/deep-merge {:a {:b {:c 1}}}\n {:a {:b {:c 2 :d 2}}}\n {:a {:f 3}}))))\n (testing \"collections as values\"\n (is (= {:a [4 5 6]}\n (configuring/deep-merge {:a [1 2 3]} {:a [4 5 6]}))))\n\n (testing \"reseting values\"\n (is (= {:a nil}\n (configuring/deep-merge {:a 1} {:a nil}))))\n\n (testing \"missing things\"\n (is (= {:a 1}\n (configuring/deep-merge {:a 1} {})))\n (is (= nil\n (configuring/deep-merge {:a 1} nil)))\n (is (= {:a 1}\n (apply configuring/deep-merge (filter some? [{:a 1} nil]))))))"
},
{
"path": "test/de/otto/tesla/stateful/handler_test.clj",
"content": "(ns de.otto.tesla.stateful.handler-test\n (:require [clojure.test :refer :all]\n [de.otto.tesla.stateful.handler :as handler]\n [com.stuartsierra.component :as component]))\n\n(deftest registering-handlers\n (testing \"should register a handler and return a single handling-function\"\n (let [handler (-> (handler/new-handler) (component/start))]\n (handler/register-handler handler (fn [r] (when (= r :ping) :pong)))\n (handler/register-handler handler (fn [r] (when (= r :pong) :ping)))\n (is (= 2 (count @(:registered-handlers handler))))\n (is (= :pong ((handler/handler handler) :ping)))\n (is (= :ping ((handler/handler handler) :pong))))))\n\n\n"
},
{
"path": "test/de/otto/tesla/stateful/health_test.clj",
"content": "(ns de.otto.tesla.stateful.health-test\n (:require [clojure.test :refer :all]\n [de.otto.tesla.stateful.health :as health]\n [de.otto.tesla.util.test-utils :as u]\n [de.otto.tesla.system :as system]\n [de.otto.tesla.stateful.handler :as handler]\n [ring.mock.request :as mock]\n [de.otto.goo.goo :as goo]))\n\n(defn- serverless-system [runtime-config]\n (dissoc\n (system/base-system runtime-config)\n :server))\n\n(deftest ^:unit should-turn-unhealthy-when-locked\n (u/with-started [started (serverless-system {})]\n (testing \"it is still healthy when not yet locked\"\n (let [handlers (handler/handler (:handler started))\n response (handlers (mock/request :get \"/health\"))]\n (are [key value] (= value (get response key))\n :body \"HEALTHY\"\n :status 200)\n (is (= 1.0 (-> :health/locked ((goo/snapshot)) (.get))))))\n\n (testing \"when locked, it is unhealthy\"\n (let [handlers (handler/handler (:handler started))\n _ (health/lock-application (:health started))\n response (handlers (mock/request :get \"/health\"))]\n (are [key value] (= value (get response key))\n :body \"UNHEALTHY\"\n :status 423)\n (is (= 0.0 (-> :health/locked ((goo/snapshot)) (.get))))))))\n\n(deftest ^:integration should-serve-health-under-configured-url\n (testing \"use the default url\"\n (u/with-started [started (serverless-system {})]\n (let [handlers (handler/handler (:handler started))]\n (is (= (:body (handlers (mock/request :get \"/health\")))\n \"HEALTHY\"\n )))))\n\n (testing \"use the configuration url\"\n (u/with-started [started (serverless-system {:health-url \"/my-health\"})]\n (let [handlers (handler/handler (:handler started))]\n (is (= (:body (handlers (mock/request :get \"/my-health\")))\n \"HEALTHY\"))))))\n\n\n\n\n\n\n"
},
{
"path": "test/de/otto/tesla/stateful/keep_alive_test.clj",
"content": "(ns de.otto.tesla.stateful.keep-alive-test\n (:require\n [clojure.test :refer [deftest testing is]]\n [de.otto.tesla.stateful.keep-alive :as kalive]\n [de.otto.tesla.util.test-utils :refer [eventually with-started]]\n [clojure.tools.logging :as log]))\n\n(deftest starting-and-stopping-the-keepalive-component\n (testing \"should start and stop keepalive-thread\"\n (let [state (atom :not-started)]\n (with-redefs [kalive/enter-keep-alive (fn []\n (log/info \"ENTER test keepalive\")\n (reset! state :entered))\n kalive/exit-keep-alive (fn []\n (log/info \"EXIT test keepalive\")\n (reset! state :exited))]\n (is (= :not-started @state))\n (with-started [_ (kalive/new-keep-alive)]\n (eventually (= :entered @state)))\n (eventually (= :exited @state))))))\n"
},
{
"path": "test/de/otto/tesla/stateful/scheduler_test.clj",
"content": "(ns de.otto.tesla.stateful.scheduler-test\n (:require [clojure.test :refer :all]\n [de.otto.tesla.system :as system]\n [de.otto.tesla.stateful.scheduler :as schedule]\n [de.otto.tesla.util.test-utils :as u]\n [overtone.at-at :as at]\n [de.otto.tesla.util.test-utils :refer [eventually]]\n [com.stuartsierra.component :as c]\n [de.otto.tesla.stateful.handler :as handler]\n [ring.mock.request :as mock]\n [clojure.data.json :as json]\n [de.otto.tesla.stateful.scheduler :as scheduler])\n (:import (java.util.concurrent ScheduledThreadPoolExecutor)))\n\n(defn- serverless-system [runtime-config]\n (-> (system/base-system runtime-config)\n (dissoc :server)\n (assoc :scheduler (c/using (schedule/new-scheduler) [:config :app-status]))))\n\n(deftest ^:unit should-call-function-at-scheduled-rate\n (testing \"Function gets called every 20 ms\"\n (u/with-started [system (serverless-system {:scheduler {:cpu-count 1}})]\n (let [scheduler (:scheduler system)\n calls (atom 0)]\n (at/every 20 #(swap! calls inc) (:pool scheduler))\n (eventually (= @calls 3)))))\n\n (testing \"Function gets called every 20 ms with initial delay\"\n (u/with-started [system (serverless-system {:scheduler {:cpu-count 1}})]\n (let [scheduler (:scheduler system)\n calls (atom 0)]\n (at/every 20 #(swap! calls inc) (:pool scheduler) :initial-delay 20)\n (eventually (= @calls 2)))))\n\n (testing \"Function gets called every 20 ms AFTER the function last returned\"\n (u/with-started [system (serverless-system {:scheduler {:cpu-count 1}})]\n (let [scheduler (:scheduler system)\n calls (atom 0)]\n (at/interspaced 20 #((Thread/sleep 10) (swap! calls inc)) (:pool scheduler))\n (eventually (= @calls 1))))))\n\n(defn assert-map-args! [args-assert-val]\n (fn [& {:as args}] (is (= args args-assert-val))))\n\n(deftest ^:unit configuring-the-schedule\n (testing \"should pass nr cpus to pool if specified\"\n (let [config {:scheduler {:cpu-count 2\n :stop-delayed? false\n :stop-periodic? true}}]\n (with-redefs [at/stop-and-reset-pool! (constantly nil)\n at/mk-pool (assert-map-args! {:cpu-count 2\n :stop-delayed? false\n :stop-periodic? true})]\n (u/with-started [system (serverless-system config)]\n (is ())))))\n\n (testing \"should pass nothing to pool if nothing is specified\"\n (let [config {:some-other :property}]\n (with-redefs [at/stop-and-reset-pool! (constantly nil)\n at/mk-pool (assert-map-args! {:cpu-count 0})]\n (u/with-started [system (serverless-system config)]\n (is ()))))))\n\n(deftest ^:unit scheduler-app-status\n (with-redefs [schedule/as-readable-time (constantly \"mock-time\")]\n (u/with-started [system (serverless-system {:host-name \"bar\" :server-port \"0123\"\n :scheduler {:cpu-count 2}})]\n (let [{:keys [scheduler handler]} system\n handler-fn (handler/handler handler)]\n (testing \"should register and return status-details in app-status\"\n (at/every 20 #(Thread/sleep 10) (:pool scheduler) :desc \"Job 1\")\n (at/interspaced 20 #(Thread/sleep 10) (:pool scheduler) :desc \"Job 2\")\n (eventually (= {:poolInfo {:active 0\n :poolSize 2\n :queueSize 2}\n :scheduledJobs {:1 {:createdAt \"mock-time\"\n :desc \"Job 1\"\n :initialDelay 0\n :msPeriod 20\n :scheduled? true}\n :2 {:createdAt \"mock-time\"\n :desc \"Job 2\"\n :initialDelay 0\n :msPeriod 20\n :scheduled? true}}\n :status \"OK\"}\n (-> (mock/request :get \"/status\")\n (handler-fn)\n :body\n (json/read-str :key-fn keyword)\n (get-in [:application :statusDetails :scheduler])))))))))\n\n(deftest ^:unit scheduler-default-conf\n (testing \"should startup pool with core-pool-size of 0 if nothing else is configured\"\n (u/with-started [system (serverless-system {})]\n (let [{:keys [scheduler]} system\n ^ScheduledThreadPoolExecutor thread-pool (:thread-pool @(:pool-atom (:pool scheduler)))]\n (is (= 0 (.getCorePoolSize thread-pool))))))\n\n (testing \"should startup pool with configured core-pool-size\"\n (u/with-started [system (serverless-system {:scheduler {:cpu-count 2}})]\n (let [{:keys [scheduler]} system\n ^ScheduledThreadPoolExecutor thread-pool (:thread-pool @(:pool-atom (:pool scheduler)))]\n (is (= 2 (.getCorePoolSize thread-pool)))))))\n"
},
{
"path": "test/de/otto/tesla/system_test.clj",
"content": "(ns de.otto.tesla.system-test\n (:require [clojure.test :refer :all]\n [com.stuartsierra.component :as c]\n [de.otto.tesla.util.test-utils :as u :refer [eventually]]\n [de.otto.tesla.system :as system]\n [ring.mock.request :as mock]\n [de.otto.tesla.stateful.handler :as handler]\n [de.otto.tesla.stateful.configuring :as configuring]\n [environ.core :as env]\n [overtone.at-at :as at]))\n\n(deftest ^:unit should-start-base-system-and-shut-it-down\n (testing \"start then shutdown using own method\"\n (let [system-stop-calls (atom 0)\n started (system/start (system/base-system {}))]\n (with-redefs [c/stop-system (fn [_] (swap! system-stop-calls inc))]\n (system/stop started)\n (is (= 1 @system-stop-calls))))))\n\n(defrecord BombOnStartup []\n c/Lifecycle\n (start [_self]\n (throw (Exception. \"boom!\")))\n (stop [self]\n self))\n\n(defn new-bomb-on-startup []\n (map->BombOnStartup {}))\n\n(deftest ^:unit should-shutdown-on-error-while-starting\n (let [exploding-system (assoc (system/base-system {}) :bomb (c/using (new-bomb-on-startup) [:health]))\n system-stop-calls (atom 0)]\n (with-redefs [c/stop-system (fn [_] (swap! system-stop-calls inc))]\n (system/start exploding-system)\n (is (= 1 @system-stop-calls)))))\n\n(defrecord BombEverytime []\n c/Lifecycle\n (start [_self]\n (throw (Exception. \"boom!\")))\n (stop [_self]\n (throw (Exception. \"boom!\"))))\n\n(defn new-bomb-on-everytime []\n (map->BombEverytime {}))\n\n(deftest should-exit-jvm-if-stopping-system-fails\n (let [exploding-system (assoc (system/base-system {}) :bomb (c/using (new-bomb-on-everytime) [:health]))\n system-exit-calls (atom [])]\n (with-redefs [system/exit #(swap! system-exit-calls conj %)]\n (system/start exploding-system)\n (is (= [1] @system-exit-calls)))))\n\n(defrecord BombOnShutdown []\n c/Lifecycle\n (start [self]\n self)\n (stop [_self]\n (throw (Exception. \"boom!\"))))\n\n(defn new-bomb-on-shutdown []\n (map->BombOnShutdown {}))\n\n(deftest ^:unit should-shutdown-on-error-while-stopping\n (let [exploding-system-on-stop (assoc (system/base-system {}) :bomb (c/using (new-bomb-on-shutdown) [:health]))\n system-exit-calls (atom [])]\n (with-redefs [system/exit #(swap! system-exit-calls conj %)]\n (system/stop (system/start exploding-system-on-stop))\n (is (= [1] @system-exit-calls)))))\n\n(deftest should-lock-application-on-shutdown\n (testing \"the lock is set\"\n (u/with-started\n [started (system/base-system {:wait-ms-on-stop 10})]\n (let [healthcomp (:health started)]\n (with-redefs [system/exit #(println \"System exit would be called with code \" %)]\n (system/stop started))\n (is (= @(:locked healthcomp) true)))))\n\n (testing \"it waits on stop\"\n (u/with-started\n [started (system/base-system {:wait-seconds-on-stop 1})]\n (let [has-waited (atom false)]\n (with-redefs [system/wait! (fn [_] (reset! has-waited true))\n system/exit #(println \"System exit would be called with code \" %)]\n (let [healthcomp (:health started)\n _ (system/stop started)]\n (is (= @(:locked healthcomp) true))\n (is (= @has-waited true))))))))\n\n\n(deftest ^:integration should-substitute-env-variables-while-reading\n (with-redefs [env/env {:my-custom-status-url \"/custom/status/path\" :prop-without-fallback \"some prop value\"}]\n (u/with-started [started (system/base-system {})]\n (testing \"should load the status-path property from edn\"\n (is (= \"/custom/status/path\"\n (:status-url (configuring/load-config-from-edn-files {})))))\n\n (testing \"should point to edn-configured custom status url\"\n (let [handlers (handler/handler (:handler started))\n response (handlers (mock/request :get \"/custom/status/path\"))]\n (is (= 200 (:status response)))))))\n\n (u/with-started [started (system/base-system {})]\n (testing \"should fallback to default for status path\"\n (is (= \"/status\"\n (:status-url (configuring/load-config-from-edn-files {})))))))\n\n(deftest the-scheduler-in-the-base-system\n (testing \"should schedule and execute task NOW\"\n (u/with-started [started (system/base-system {})]\n (let [work-done (atom :no-work-done)\n {:keys [scheduler]} started]\n (at/after 0 #(reset! work-done :work-done!) (:pool scheduler))\n (eventually (= :work-done! @work-done))))))\n"
},
{
"path": "test/de/otto/tesla/util/env_var_reader_test.clj",
"content": "(ns de.otto.tesla.util.env-var-reader-test\n (:require [de.otto.tesla.util.env_var_reader :as env-reader]\n [clojure.test :refer :all]\n [environ.core :as env]))\n\n(deftest ^:unit should-map-keys-to-sanitized-keywords\n (with-redefs [env/env {\"my-var-key\" \"my-var-value\"}]\n (is (= \"my-var-value\"\n (env-reader/read-env-var [\"my-var-key\"])))))\n\n(deftest ^:unit should-use-fallback-if-env-does-not-have-key\n (with-redefs [env/env {}]\n (is (= \"default\"\n (env-reader/read-env-var [\"my-var-key\" \"default\"])))))\n\n(deftest ^:unit should-return-empty-if-no-fallback-and-key-not-in-env\n (with-redefs [env/env {}]\n (is (= \"\"\n (env-reader/read-env-var [\"my-var-key\"])))))"
},
{
"path": "test/de/otto/tesla/util/keyword_test.clj",
"content": "(ns de.otto.tesla.util.keyword-test\n (:require [clojure.test :refer :all]\n [de.otto.tesla.util.keyword :as kwutil]))\n\n(deftest ^:unit should-map-keys-to-sanitized-keywords\n (is (= (kwutil/sanitize-keywords {\"thats.a.property\" \"a value\"})\n {:thats-a-property \"a value\"})))"
},
{
"path": "test/de/otto/tesla/util/sanitize_test.clj",
"content": "(ns de.otto.tesla.util.sanitize-test\n (:require [clojure.test :refer :all]\n [de.otto.tesla.util.sanitize :as san]))\n\n(deftest ^:unit should-sanitize-passwords\n (is (= (san/hide-passwds {:somerandomstuff \"not-so-secret\"\n :somerandomstuff-passwd-somerandomstuff \"secret\"\n :somerandomstuff-pw-somerandomstuff \"longersecret\"\n :my-private-stuff \"sonotpublic\"\n :my-secret-stuff \"psstsecret\"\n :nested {:some-passwd \"secret\"}})\n {:somerandomstuff \"not-so-secret\"\n :somerandomstuff-passwd-somerandomstuff \"***\"\n :somerandomstuff-pw-somerandomstuff \"***\"\n :my-private-stuff \"***\"\n :my-secret-stuff \"***\"\n :nested {:some-passwd \"***\"}\n })))\n"
},
{
"path": "test/de/otto/tesla/util/test_utils_test.clj",
"content": "(ns de.otto.tesla.util.test-utils-test\n (:require\n [clojure.test :refer :all]\n [de.otto.tesla.util.test-utils :as u]\n [com.stuartsierra.component :as c]))\n\n(deftest eventually\n (testing \"should eventually get the right random number\"\n (u/eventually (= 1 (rand-int 2))))\n\n (testing \"should eventually return the right response\"\n (let [start-time (System/currentTimeMillis)]\n (u/eventually (= :expected-response (#(let [waited-200msec? (> (- (System/currentTimeMillis) start-time) 200)]\n (if waited-200msec?\n :expected-response\n :wrong-response))))))))\n\n(defrecord StartableComponent [state]\n c/Lifecycle\n (start [self] (reset! state :started) self)\n (stop [self] (reset! state :stopped) self))\n\n(deftest with-started\n (testing \"should start and stop something\"\n (let [state (atom :not-running)]\n (is (= :not-running @state))\n (u/with-started [started (->StartableComponent state)]\n (is (= :started @state)))\n (is (= :stopped @state)))))\n\n(deftest testing-the-mock-request\n (testing \"should create mock-request\"\n (is (= (u/mock-request :get \"url\" {})\n {:headers {\"host\" \"localhost\"}\n :query-string nil\n :remote-addr \"localhost\"\n :request-method :get\n :scheme :http\n :server-name \"localhost\"\n :server-port 80\n :uri \"url\"})))\n (testing \"should create mock-request\"\n (is (= (u/mock-request :get \"url\" {:headers {\"content-type\" \"application/json\"}})\n {:headers {\"host\" \"localhost\"\n \"content-type\" \"application/json\"}\n :query-string nil\n :remote-addr \"localhost\"\n :request-method :get\n :scheme :http\n :server-name \"localhost\"\n :server-port 80\n :uri \"url\"}))))\n"
},
{
"path": "test-resources/local.edn",
"content": "{:server-port \"9991\"\n :hostname #ts/env [:host-name \"localhost\"]\n :status-url #ts/env [:my-custom-status-url \"/status\"]\n :prop-without-fallback #ts/env [:prop-without-fallback]\n :metric {:console {:interval-in-s 120}}\n :foo {:edn \"baz\"}\n :very :local}\n"
},
{
"path": "test-resources/local.properties",
"content": "foo.prop=baz\nserver.port=9991"
},
{
"path": "test-resources/logback-test.xml",
"content": "\n\n tesla\n\n\n\n \n true\n \n %d{HH:mm:ss.SSS} %-5p %c{5} %t \"%m\"%n\n \n \n\n\n \n \n \n\n\n"
},
{
"path": "test-resources/private.edn",
"content": "; this is checked in for testing reasons.\n; In your projects you should add 'private.edn' to .gitignore.\n\n{:very :private}"
},
{
"path": "test-resources/test.edn",
"content": "{:health-url \"/test/health\"\n :foo {:local true}}"
},
{
"path": "test-resources/version.properties",
"content": "version=test.version\ncommit=test.githash"
},
{
"path": "test-utils/de/otto/tesla/util/test_utils.clj",
"content": "(ns de.otto.tesla.util.test-utils\n (:require\n [com.stuartsierra.component :as comp]\n [ring.mock.request :as mock]))\n\n(defmacro eventually\n \"Generic assertion macro, that waits for a predicate test to become true.\n 'form' is a predicate test, that clojure.test/is can understand\n 'timeout': optional; in ms; how long to wait in total (defaults to 5000)\n 'interval' optional; in ms; how long to pause between tries (defaults to 10)\n\n Example:\n Since this will fail half of the time ...\n (is (= 1 (rand-int 2)))\n\n ... use this:\n (eventually (= 1 (rand-int 2)))\"\n [form & {:keys [timeout interval]\n :or {timeout 5000\n interval 10}}]\n `(let [start-time# (System/currentTimeMillis)]\n (loop []\n (let [last-stats# (atom nil)\n pass?# (with-redefs [clojure.test/do-report (fn [s#] (reset! last-stats# s#))]\n (clojure.test/is ~form))\n took-too-long?# (> (- (System/currentTimeMillis) start-time#) ~timeout)]\n (if (or pass?# took-too-long?#)\n (clojure.test/do-report @last-stats#)\n (do\n (Thread/sleep ~interval)\n (recur)))))))\n\n(defmacro with-started\n \"bindings => [name init ...]\n\n Evaluates body in a try expression with names bound to the values\n of the inits after (.start init) has been called on them. Finally\n a clause calls (.stop name) on each name in reverse order.\"\n [bindings & body]\n (if (and\n (vector? bindings) \"a vector for its binding\"\n (even? (count bindings)) \"an even number of forms in binding vector\")\n (cond\n (= (count bindings) 0) `(do ~@body)\n (symbol? (bindings 0)) `(let [~(bindings 0) (comp/start ~(bindings 1))]\n (try\n (with-started ~(subvec bindings 2) ~@body)\n (finally\n (comp/stop ~(bindings 0)))))\n :else (throw (IllegalArgumentException.\n \"with-started-system only allows Symbols in bindings\")))\n (throw (IllegalArgumentException.\n \"not a vector or bindings-count is not even\"))))\n\n(defn mock-request\n \"merges additional arguments into a mock-request\"\n [method url args]\n (let [request (mock/request method url)]\n (merge-with merge request args)))\n"
}
]