[ { "path": ".gitignore", "content": "tmp\neclipse/bin\n.cproject\n.project\nbin\nlib\ndeps\n" }, { "path": "CONTRIBUTING.md", "content": "## Contributing to Ekam\n\n### Building\n\nUse `make continuous` to build Ekam continuously and watch for changes. For LSP support in Visual Studio Code, run `make setup-vscode`.\n\n`make continuous` requires you already have an install of Ekam. You can install Ekam by building it once (`make`) and then installing `bin/ekam{,-client,-langserve}` to a directory in PATH.\n\nNote that `g++` tends to generate spurious warnings, so you may want to use `make continuous CXX=clang++` instead.\n\n### Overview\n\nEkam has multiple components.\n- `ekam`: This is the main binary. It actually builds the code, and uses capnp to communicate with the other binaries.\n- `ekam-client`: This is a small tool to view the build output from `ekam`.\n- `ekam-langserve`: This is a Language Server Protocol client that communicates with `ekam`.\n- `ekam-bootstrap`: This is the same as `ekam`, but doesn't support the remote capnp protocol. As a result, some flags (like `-n`) aren't supported, and prebuilt capnp clients won't be able to connect.\n\n`ekam-bootstrap` is built directly by `make`. All other components are built by `ekam-bootstrap`.\n" }, { "path": "CONTRIBUTORS", "content": "The following people have made large code contributions to this repository.\nThose contributions are copyright the respective authors and licensed by them\nunder the same Apache 2.0 license terms as the rest of the library.\n\nKenton Varda : Primary Author\nGoogle, Inc.: Employed Kenton when he wrote much of this code, and thus owns\n copyright. Ekam was a 20% project as is not in any way endorsed by Google.\n\n" }, { "path": "LICENSE", "content": " Apache License\n Version 2.0, January 2004\n http://www.apache.org/licenses/\n\n TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n 1. Definitions.\n\n \"License\" shall mean the terms and conditions for use, reproduction,\n and distribution as defined by Sections 1 through 9 of this document.\n\n \"Licensor\" shall mean the copyright owner or entity authorized by\n the copyright owner that is granting the License.\n\n \"Legal Entity\" shall mean the union of the acting entity and all\n other entities that control, are controlled by, or are under common\n control with that entity. For the purposes of this definition,\n \"control\" means (i) the power, direct or indirect, to cause the\n direction or management of such entity, whether by contract or\n otherwise, or (ii) ownership of fifty percent (50%) or more of the\n outstanding shares, or (iii) beneficial ownership of such entity.\n\n \"You\" (or \"Your\") shall mean an individual or Legal Entity\n exercising permissions granted by this License.\n\n \"Source\" form shall mean the preferred form for making modifications,\n including but not limited to software source code, documentation\n source, and configuration files.\n\n \"Object\" form shall mean any form resulting from mechanical\n transformation or translation of a Source form, including but\n not limited to compiled object code, generated documentation,\n and conversions to other media types.\n\n \"Work\" shall mean the work of authorship, whether in Source or\n Object form, made available under the License, as indicated by a\n copyright notice that is included in or attached to the work\n (an example is provided in the Appendix below).\n\n \"Derivative Works\" shall mean any work, whether in Source or Object\n form, that is based on (or derived from) the Work and for which the\n editorial revisions, annotations, elaborations, or other modifications\n represent, as a whole, an original work of authorship. For the purposes\n of this License, Derivative Works shall not include works that remain\n separable from, or merely link (or bind by name) to the interfaces of,\n the Work and Derivative Works thereof.\n\n \"Contribution\" shall mean any work of authorship, including\n the original version of the Work and any modifications or additions\n to that Work or Derivative Works thereof, that is intentionally\n submitted to Licensor for inclusion in the Work by the copyright owner\n or by an individual or Legal Entity authorized to submit on behalf of\n the copyright owner. For the purposes of this definition, \"submitted\"\n means any form of electronic, verbal, or written communication sent\n to the Licensor or its representatives, including but not limited to\n communication on electronic mailing lists, source code control systems,\n and issue tracking systems that are managed by, or on behalf of, the\n Licensor for the purpose of discussing and improving the Work, but\n excluding communication that is conspicuously marked or otherwise\n designated in writing by the copyright owner as \"Not a Contribution.\"\n\n \"Contributor\" shall mean Licensor and any individual or Legal Entity\n on behalf of whom a Contribution has been received by Licensor and\n subsequently incorporated within the Work.\n\n 2. Grant of Copyright License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n copyright license to reproduce, prepare Derivative Works of,\n publicly display, publicly perform, sublicense, and distribute the\n Work and such Derivative Works in Source or Object form.\n\n 3. Grant of Patent License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n (except as stated in this section) patent license to make, have made,\n use, offer to sell, sell, import, and otherwise transfer the Work,\n where such license applies only to those patent claims licensable\n by such Contributor that are necessarily infringed by their\n Contribution(s) alone or by combination of their Contribution(s)\n with the Work to which such Contribution(s) was submitted. If You\n institute patent litigation against any entity (including a\n cross-claim or counterclaim in a lawsuit) alleging that the Work\n or a Contribution incorporated within the Work constitutes direct\n or contributory patent infringement, then any patent licenses\n granted to You under this License for that Work shall terminate\n as of the date such litigation is filed.\n\n 4. Redistribution. You may reproduce and distribute copies of the\n Work or Derivative Works thereof in any medium, with or without\n modifications, and in Source or Object form, provided that You\n meet the following conditions:\n\n (a) You must give any other recipients of the Work or\n Derivative Works a copy of this License; and\n\n (b) You must cause any modified files to carry prominent notices\n stating that You changed the files; and\n\n (c) You must retain, in the Source form of any Derivative Works\n that You distribute, all copyright, patent, trademark, and\n attribution notices from the Source form of the Work,\n excluding those notices that do not pertain to any part of\n the Derivative Works; and\n\n (d) If the Work includes a \"NOTICE\" text file as part of its\n distribution, then any Derivative Works that You distribute must\n include a readable copy of the attribution notices contained\n within such NOTICE file, excluding those notices that do not\n pertain to any part of the Derivative Works, in at least one\n of the following places: within a NOTICE text file distributed\n as part of the Derivative Works; within the Source form or\n documentation, if provided along with the Derivative Works; or,\n within a display generated by the Derivative Works, if and\n wherever such third-party notices normally appear. The contents\n of the NOTICE file are for informational purposes only and\n do not modify the License. You may add Your own attribution\n notices within Derivative Works that You distribute, alongside\n or as an addendum to the NOTICE text from the Work, provided\n that such additional attribution notices cannot be construed\n as modifying the License.\n\n You may add Your own copyright statement to Your modifications and\n may provide additional or different license terms and conditions\n for use, reproduction, or distribution of Your modifications, or\n for any such Derivative Works as a whole, provided Your use,\n reproduction, and distribution of the Work otherwise complies with\n the conditions stated in this License.\n\n 5. Submission of Contributions. Unless You explicitly state otherwise,\n any Contribution intentionally submitted for inclusion in the Work\n by You to the Licensor shall be under the terms and conditions of\n this License, without any additional terms or conditions.\n Notwithstanding the above, nothing herein shall supersede or modify\n the terms of any separate license agreement you may have executed\n with Licensor regarding such Contributions.\n\n 6. Trademarks. This License does not grant permission to use the trade\n names, trademarks, service marks, or product names of the Licensor,\n except as required for reasonable and customary use in describing the\n origin of the Work and reproducing the content of the NOTICE file.\n\n 7. Disclaimer of Warranty. Unless required by applicable law or\n agreed to in writing, Licensor provides the Work (and each\n Contributor provides its Contributions) on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n implied, including, without limitation, any warranties or conditions\n of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n PARTICULAR PURPOSE. You are solely responsible for determining the\n appropriateness of using or redistributing the Work and assume any\n risks associated with Your exercise of permissions under this License.\n\n 8. Limitation of Liability. In no event and under no legal theory,\n whether in tort (including negligence), contract, or otherwise,\n unless required by applicable law (such as deliberate and grossly\n negligent acts) or agreed to in writing, shall any Contributor be\n liable to You for damages, including any direct, indirect, special,\n incidental, or consequential damages of any character arising as a\n result of this License or out of the use or inability to use the\n Work (including but not limited to damages for loss of goodwill,\n work stoppage, computer failure or malfunction, or any and all\n other commercial damages or losses), even if such Contributor\n has been advised of the possibility of such damages.\n\n 9. Accepting Warranty or Additional Liability. While redistributing\n the Work or Derivative Works thereof, You may choose to offer,\n and charge a fee for, acceptance of support, warranty, indemnity,\n or other liability obligations and/or rights consistent with this\n License. However, in accepting such obligations, You may act only\n on Your own behalf and on Your sole responsibility, not on behalf\n of any other Contributor, and only if You agree to indemnify,\n defend, and hold each Contributor harmless for any liability\n incurred by, or claims asserted against, such Contributor by reason\n of your accepting any such warranty or additional liability.\n\n END OF TERMS AND CONDITIONS\n\n Copyright 2015 Sandstorm Development Group, Inc.\n\n Licensed under the Apache License, Version 2.0 (the \"License\");\n you may not use this file except in compliance with the License.\n You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software\n distributed under the License is distributed on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n See the License for the specific language governing permissions and\n limitations under the License.\n\n" }, { "path": "Makefile", "content": "# Ekam Build System\n# Author: Kenton Varda (kenton@sandstorm.io)\n# Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n.SUFFIXES:\n.PHONY: all install clean deps continuous setup-vscode\n\n# You may override the following vars on the command line to suit\n# your config.\nCXX=g++\nCXXFLAGS=-O2 -Wall\nPARALLEL=$(shell nproc)\n# for `make continuous`\nEKAM=ekam\n\ndefine color\n @printf '\\033[0;34m==== $1 ====\\033[0m\\n'\nendef\n\nall: bin/ekam\n\nbin/ekam: bin/ekam-bootstrap | deps\n\t$(call color,building ekam with ekam)\n\t@rm -f bin/ekam\n\t@CXX=\"$(CXX)\" CXXFLAGS=\"-std=c++14 $(CXXFLAGS) -pthread\" LIBS=\"-pthread\" bin/ekam-bootstrap -j$(PARALLEL)\n\t@test -e bin/ekam && printf \"=====================================================\\nSUCCESS\\nOutput is at bin/ekam\\n=====================================================\\n\"\n\n# NOTE: Needs a full install of Ekam instead of the bootstrap so that LSP is available.\n# To avoid recompiling 3 times, this requires the user to have Ekam already installed.\ncontinuous:\n\t$(call color,building ekam with ekam continuously)\n\t@CXX=\"$(CXX)\" CXXFLAGS=\"-std=c++14 $(CXXFLAGS) -pthread\" LIBS=\"-pthread\" $(EKAM) -j$(PARALLEL) -n :41315 -c\n\nsetup-vscode: vscode/vscode-ekam-0.2.0.vsix\n\tcode --install-extension $<\n\nvscode/vscode-ekam-0.2.0.vsix:\n\tcd vscode && npm install\n\tcd vscode && npm run package\n\ndeps: deps/capnproto\n\ndeps/capnproto:\n\t$(call color,downloading capnproto)\n\t@mkdir -p deps\n\tgit clone https://github.com/capnproto/capnproto.git deps/capnproto\n\nSOURCES=$(shell cd src; find base os ekam -name '*.cpp' | \\\n grep -v KqueueEventManager | grep -v PollEventManager | \\\n grep -v ProtoDashboard | grep -v ekam-client | grep -v _test)\n\nHEADERS=$(shell find src/base src/os src/ekam -name '*.h')\n\n# Use a subdirectory for bootstrapping so the object files don't overlap with the Ekam build.\nOBJ_DIR := tmp/bootstrap\nOBJECTS=$(addprefix $(OBJ_DIR)/, $(SOURCES:.cpp=.o))\n\n$(OBJ_DIR)/%.o: src/%.cpp $(HEADERS)\n\t@echo $(HEADERS)\n\t@mkdir -p $(@D)\n\t$(CXX) -Isrc -fPIC -std=c++14 -pthread -o $@ -c $<\n\nbin/ekam-bootstrap: $(OBJECTS)\n\t$(call color,compiling bootstrap ekam)\n\t@mkdir -p bin\n\t$(CXX) -Isrc -std=c++14 -pthread $(OBJECTS) -o $@\n\nclean:\n\trm -rf bin lib tmp $(OBJ_DIR)\n\n" }, { "path": "README.md", "content": "# Ekam Build System\n\nEkam (\"make\" backwards) is a build system which automatically figures out what to build and how to build it purely based on the source code. No separate \"makefile\" is needed.\n\nEkam works by exploration. For example, when it encounters a file ending in \".cpp\", it tries to compile the file, intercepting system calls to find out its dependencies (e.g. included headers). If some of these are missing, Ekam continues to explore until it finds headers matching them. When Ekam builds an object file and discovers that it contains a \"main\" symbol, it tries to link it, searching for other object files to satisfy all symbol references therein. When Ekam sees a test, it runs the test, again intercepting system calls to dynamically discover what inputs the test needs (which may not have been built yet). Ekam can be extended to understand new file types by writing simple shell scripts telling it what to do with them.\n\nThus Ekam is, in fact, Make in reverse. Make starts by reading a Makefile, sees what executables it wants to build, then from there figures out what source files need to be compiled to link into them, then compiles them. Ekam starts by looking for source files to compile, then determines what executables it can build from them, and, in the end, might output a Makefile describing what it did.\n\nEkam is a work in progress.\n\n## Building Ekam\n\n### Warning\n\nEkam is an experimental project that is not ready for wide use.\n\nThat said, I (Kenton) have successfully used Ekam as my primary build system throughout the development of [Cap'n Proto](https://capnproto.org), [Sandstorm](https://sandstorm.io), and the [Cloudflare Workers](https://developers.cloudflare.com/workers/about/) runtime.\n\n### Supported Platforms\n\nAt this time, Ekam only runs on Linux. It requires GCC 5+ or Clang 4+, as it uses C++14 features.\n\nIn the past, Ekam worked on FreeBSD and Max OSX, but the code to support that atrophied and was eventually deleted. Ekam uses a lot of OS-specific hacks and so is unlikely to work on any platform which is not explicitly supported.\n\nWe'd like to see other platforms supported, but Ekam's primary user and maintainer right now ([Sandstorm.io](https://sandstorm.io)) is itself highly Linux-specific, so there is not much pressure. (Let us know if you want to help.)\n\n### Bootstrapping the build\n\nDownload and compile Ekam like so:\n\n git clone https://github.com/capnproto/ekam.git\n cd ekam\n make\n\nIf successful, Ekam should have built itself, with the output binary at \"bin/ekam\".\n\nYes, we use make in order to bootstrap Ekam, mostly just because it's slightly nicer than a shell script.\n\n### Compiling Ekam with Ekam\n\nCompiling Ekam requires the GCC flag `-std=gnu++0x` to enable C++11 features, but currently there is no way for the code itself to specify compiler flags that it requires. You can only specify them via environment variable. So, to build Ekam with Ekam, type this command at the top of the Ekam repository:\n\n CXXFLAGS=-std=gnu++0x ekam -j4\n\nThe `-j4` tells Ekam to run up to four tasks at once. You may want to adjust this number depending on how many CPU cores you have.\n\nNote that Ekam looks for a directory called `src` within the current directory, and scans it for source code. The Ekam source repository is already set up with such a `src` subdirectory containing the Ekam code. You could, however, place the entire Ekam repository _inside_ some other directory called `src`, and then run Ekam from the directory above that, and it will still find the code. The Protocol Buffers instructions below will take advantage of this to create a directory tree containing both Ekam and protobufs.\n\nEkam places its output in siblings of `src` called `tmp` (for intermediate files), `bin` (for output binaries), `lib` (for output libraries, although currently Ekam doesn't support building libraries), etc. These are intended to model Unix directory tree conventions.\n\n## Continuous Building\n\nIf you invoke Ekam with the `-c` option, it will watch the source tree for changes and rebuild derived files as needed. In this way, you can simply leave Ekam running while you work on your code, and get information about errors almost immediately on saving.\n\nNote that continuous building is the only way to do incremental builds with Ekam -- any time you run a new Ekam process, it always starts from scratch. I generally just leave Ekam running in a console window 24/7.\n\n## IDE plugins and other external clients\n\nEkam can, while running, export a network interface which allows other programs to query the state of the build, including receiving the task tree and error logs.\n\n### Ekam Server\n\nIf you pass the `-n` flag to Ekam, it will listen for connections on a port and stream build status updates to anyone who connects. Invoke like:\n\n ekam -n :41315\n\n### Ekam Client\n\n`ekam-client` is a very simple program that prints an Ekam build status stream to the console exactly as Ekam itself does. Currently `ekam-client` doesn't actually know how to create a network connection but instead reads the stream from standard input, so you can invoke it like this:\n\n nc localhost 41315 | ekam-client\n\n`ekam-client` is mostly just a tech demo, since it displays the same info that is already visible in the console where Ekam itself is running.\n\n### Visual Studio Code Plugin\n\nThe `vscode` directory in your Ekam repository contains source code for a [Visual Studio Code](https://code.visualstudio.com/) plugin. [See its README for more info.](vscode/README.md)\n\n### Qt Creator Plugin\n\nThe `qtcreator` directory in the Ekam repository contains source code for a [Qt Creator](http://qt-project.org/wiki/Category:Tools::QtCreator) plugin. Qt Creator is an open source C++ IDE that, despite its name, is quite nice even for non-Qt code.\n\nTo build the Qt Creator plugin, you'll want to download the Qt Creator source code release and build it from scratch. Then, go into the `qtcreator` directory in the Ekam repo and do the following:\n\n export QTC_SOURCE=\n qmake\n make\n\nThis should build the plugin and install it directly into your Qt Creator build.\n\nNow you can start Qt Creator and choose the \"Ekam Actions\" view in the navigation frame. The plugin connects to a local Ekam instance running on port 41315; so, start ekam like this:\n\n ekam -c -n :41315\n\nThe plugin will also add error markers to you source code, visible in the issue tracker. Double-clicking on a failed rule in the tree view will navigate the issues view to the first message from that action, which you can in turn use to navigate to the source location of the error.\n\n## Using Ekam in your own code\n\n### Preferred project layout\n\nAn Ekam project directory must contain a directory called `src` which contains all source code. Ekam will ignore everything other than this `src` directory.\n\nEkam will create sibling directories called `tmp`, `bin`, and `lib`; you probably shouldn't have any files in those directories to start, because it's nice to be able to `rm -rf` them to clean.\n\n### Import the rule files\n\nIn order for Ekam to be able to build a project, it needs to find its rule files, which are under `src/ekam/rules` in Ekam's own source code. You will want to symlink this directory into your project's source tree somewhere. It does not have to be anywhere in particular, but `src/ekam-rules` is probably a good bet.\n\n### Compiler flags\n\nYou can set the following environment variables to control how Ekam compiles your code:\n\n* `CXX`: Sets the C++ compiler, e.g. `CXX=clang++`.\n* `CXXFLAGS`: Sets C++ compilation flags, e.g. `CXXFLAGS=-std=c++11 -O2 -Wall`.\n* `LIBS`: Sets linker flags, e.g. `LIBS=-lsodium -lz`\n\n### Building binaries\n\nWhen Ekam finds or compiles a `.o` file that defines the symbol `main`, it will attempt to link the file with other `.o` files defining all needed symbols (transitively) in order to produce a binary. The binary takes the name of the `.o` with the extension removed, e.g. `foo.o` produces a binary `foo`.\n\nThe binary is initially just dropped into the `tmp` directory, which mirrors the `src` directory. For example, `src/foo/bar.c++` will be compiled to `src/foo/bar.o`, which will link (if it defines `main`) into `src/foo/bar`. If you'd like for the binary to be output to the `bin` directory, you must declare a manifest file with the extension `.ekam-manifest`. This file is a simple text file where each line names a file and its output directory. For example:\n\n bar bin\n\nThis says: \"Once you've built `bar` (within this directory), copy it into `bin`.\"\n\nYou can also choose to rename the file when installing:\n\n bar bin/baz\n\nThis says: \"Once you've built `bar` (within this directory), copy it to `bin/baz`.\"\n\n### Building libraries\n\nCurrently, Ekam does not support building libraries. This seems complicated to support since there's no automated way for Ekam to decide what makes a good library. You'd need to declare some set of modules representing the public interface, which is a bit sad.\n\nFor now, Ekam is suitable for projects where the final output is a binary. A \"library\" in Ekam is just a directory that contains some code. It does not build to a `.a` or `.so` file; instead, the objects may be directly linked into binaries found in other directories as needed.\n\n### Handling Magic Singleton Registries\n\nThere is a common pattern in C++ code in which a particular file's symbols are not referenced from other files, but instead the file registers itself in some global registry at program startup which makes its functionality discoverable to the rest of the program.\n\nBy default, this pattern does not work with Ekam, because Ekam has no way to tell which of these magic self-registering files should be linked into which binaries, since their symbols are not referenced from other `.o` files.\n\nTo solve this problem, don't. This design pattern sucks and you should not use it. Rewrite your code so that its functionality is not dependent on what objects were specified on the link line. For example, have your `main()` function explicitly call `registerFoo()`, `registerBar()`, etc., for all the magic modules you want registered, ideally passing a registry object to each function so you don't need a singleton registry.\n\nSee [Singletons Considered Harmful](http://www.object-oriented-security.org/lets-argue/singletons) for extended discussion.\n\n(We make a special exception for test frameworks, described below.)\n\n### Debugging\n\nIf you want `gdb` to be able to find your code when debugging Ekam-built binaries, you should set up a couple symlinks as follows:\n\n mkdir ekam-provider\n ln -s ../src ekam-provider/canonical\n ln -s ../src ekam-provider/c++header\n\nTo understand the reason for these symlinks, see the explanation of `intercept.so` later in this document.\n\n### Tests\n\nIf Ekam builds a binary which ends in `-test` or `_test`, it will run that binary and report pass/fail depending on the exit status. Passing tests are marked in green in the output (whereas regular successful build actions are blue) in order to help you develop a pavlovian attachment to writing tests and seeing them pass.\n\nEkam has additional built-in support for two test frameworks: Google Test and KJ tests. When Ekam compiles a source file which declares test cases using one of these frameworks (e.g. using Google Test's `TEST` or `TEST_F` macros, or KJ's `KJ_TEST` macro) but without a `main` function, it will automatically link against the respective framework's test runner (which supplies `main`) in order to produce a test binary. Note that the detection of test declarations is based on linker symbols, not on scanning the source code, so don't worry if you've declared your own wrapper macros.\n\nIn order for Google Test or KJ test integration to work, the respective test framework's code must be in your source tree as a dependency (see below).\n\nNote that tests are run with `intercept.so` injected, which has implications if your test does any filesystem access. See the explanation of `intercept.so` later in this document.\n\n### Dependencies\n\nIf your project depends on other projects, and you want to build those other projects as part of your own build (rather than require the user to install the libraries on their system), you should follow the pattern that Ekam itself does.\n\nThe basic idea is to have a directory called `deps` into which you download the repositories of your external dependencies. Then, under `src`, place symlinks that deep-link into `deps` and pull out the code you want.\n\nFor example, to depend on Cap'n Proto, you would do something like:\n\n mkdir deps\n git clone https://github.com/sandstorm-io/capnproto.git deps/capnproto\n ln -s ../deps/capnproto/c++/src/{kj,capnp} src\n\nNote that `deps` should not be checked into your repository. Instead, you should provide a script that people run after cloning your repo that downloads dependencies. For Ekam's own build, we handle this in the Makefile, so you can look at that for an example. We may eventually add functionality to Ekam to handle dependencies so that you no longer need a Makefile for this.\n\n**Now, here's the fun part:** If you work on a lot of different projects, instead of nesting dependencies as described above, you can symlink `deps` to `..`, such that all your git clones are siblings:\n\n rm -rf deps\n ln -s .. deps\n\nYou can do this with Ekam, for instance, if you happen to have Cap'n Proto cloned as a sibling to Ekam.\n\n### Non-Ekam-clean Dependencies\n\nIf your dependency is not organized for Ekam, you might need to link to its top-level directory rather than a source subdirectory. For instance, Google Test separates its source code into `src` and `include` directories, so if you only link in its `src` you'll miss the headers. Instead, link in the whole repo like:\n\n ln -s ../../gtest src\n\nWhen Ekam sees a directory named `include`, it adds that directory to the include path for all other code it compiles. Thus the Google Test headers will end up includable by your code.\n\nNote that projects imported this way will usually have a bunch of errors reported since they are not Ekam-clean. However, often (such as in the case of Google Test) the important parts will compile well enough to use.\n\n### Per-directory compile options\n\nTo specify some additional compiler options that should apply within a particular directory (and all children, recursively), you may create a file called `compile.ekam-flags` and populate it with simple variable assignments in shell syntax. For example:\n\n # Define FOO to 1 when compiling code in this directory.\n CXXFLAGS=$CXXFLAGS -DFOO=1\n\n`.ekam-flags` files are actually executed using `/bin/sh` just before invoking the compiler. When multiple flags files are in-scope, the flags file in the outermost directory runs first.\n\n### Cross-compiling\n\nEkam currently supports cross-compiling to multiple target architectures at once, by listing additional (non-host) architectures in the `CROSS_TARGETS` environment variable:\n\n CROSS_TARGETS=\"aarch64-linux-gnu\" ekam\n\nNotes:\n* This is designed to work e.g. with the `crossbuild-essential-*` Debian packages.\n* Ekam always compiles for the host architecture in addition to these targets. This is to support building tools like the Cap'n Proto compiler and then immediately using them in the same build.\n* Ekam assumes the inter-object dependencies are the same on all targets. This tends to mean that it work well for targetting alternate CPU architectures, but not as well for targeting other operating systems.\n* You may specify target-specific CXXFLAS and LIBS like `CXXFLAGS_aarch64_linux_gnu` and `LIBS_aarch64_linux_gnu`. If present, these completely replace the default `CXXFLAGS` and `LIBS`.\n* If any unit tests are built, Ekam will try to use qemu to run them.\n\n## Custom Rules\n\nYou may teach Ekam how to handle a new type of file -- or introduce a one-off build rule not triggered by any file -- by creating a rule file. A rule file is any executable with the extension `.ekam-rule`. Often, they are shell scripts, but this is not a requirement. Rule files can themselves be the output of other rules, so you could compile a C++ program that acts as a rule.\n\nEkam executes custom rules and then communicates with them on stdin/stdout using a simple line-based text protocol described below. The rule may also write error logs intended for the user to stderr.\n\nWhen Ekam encounters an executable with the `.ekam-rule` extension, it first runs it with no arguments in order to \"learn\" it. At that time, the program may tell ekam what kinds of files it should trigger on using the `trigger` command (below). When Ekam encounters a trigger file, it will execute the rule again with the trigger file's canonical name as the argument. Alternatively, if you just want to implement a one-off action, then the rule may simply perform that action at \"learn\" time without registering any triggers.\n\n### Canonical file names\n\nA file's \"canonical\" name is the name without the `src/` or `tmp/` prefix. Thus canonical names do not distinguish between source files and build outputs. No two files can have the same canonical name -- it is an error for a build action to output a file which exists in the `src` directory, or for two actions to produce the same file under `tmp`.\n\nYou can ask Ekam to map a canonical name to a physical disk name using the `findInput` command.\n\n### Tags\n\nEkam rules are triggered by tags. A tag is a simple string, conventionally of the format `type:value`. Tags are applied to files. For example, a `.o` file declaring the symbol `main` might be given the tag `c++symbol:main`. Rules may assign new tags to any file (source or output).\n\nEkam has some default rules that assign tags meant for broad use:\n\n* `canonical:`: Each file receives this tag, where `` is its canonical name.\n* `filetype:`: Each regular file that has a file type extension receives this tag, e.g. `filetype:.c++`.\n* `directory:*`: Each directory receives this tag.\n\n### Commands\n\nA rule may perform the following commands by writing them to standard output.\n\n* `trigger `: Used during the learning phase to tell Ekam that the rule should be executed on any file tagged with ``.\n* `verb `: Use during the learning phase to tell Ekam the rule's \"verb\", which is what is displayed to the user when the rule later runs. This should be a simple, descriptive word. For instance, for a C++ compile action, the verb is `compile`.\n* `silent`: Use during the learning phase to indicate that when this command later runs, it should not be reported to the user unless it fails. Use this to reduce noise caused by very simple commands that perform trivial actions.\n* `findInput `: Obtains the canonical name of the given file. Ekam will reply by writing one line to the rule's standard input containing the full disk path of the file (e.g. including `src/` or `tmp/`). Ekam will remember that the build action depended on this file, so if the file changes, the action will be re-run. If no match was found, Ekam will return a blank line.\n* `findProvider `: Find a file tagged with ``. If there are multiple matches, Ekam heuristically chooses the \"preferred\" one, which generally means the one closest in the directory tree to the file which triggered the rule. The path is returned as with `findInput`. Also as with `findInput`, the file is considered a dependency of the action. Ekam will re-run this action if the file changes *or* if the file Ekam chose to match `` changes.\n* `findModifiers `: Search for the file `` in the trigger file's directory and every parent up to the source root. For each place that it is found (in order starting from the greatest ancestor), return the full disk path and mark it as an input. After returning all results, return a blank line to indicate the end of the list. This command is intended for finding \"modifier\" files which specify options that should apply within a particular directory. For instance, `compile.ekam-flags` is implemented this way.\n* `noteInput `: Tells Ekam that the action depends on ``, which is a path outside of the project's source tree. For instance, `/usr/include/stdlib.h`. Currently Ekam ignores this, but in theory it could watch these files and re-run the action if they change.\n* `newOutput `: Create a new output file with the given canonical name. Ekam replies by writing the on-disk path where the file should be created to the rule's standard input.\n* `provide `: Tag `` (a canonical name) with ``. The file must be a known input our output of this rule; i.e. it must have been the subeject of a previous call to `findInput`, `findProvider`, or `newOutput`.\n* `install `: Take the canonical filename `` and copy it to ``, where `` should start with `bin/`, `lib/`, etc.\n* `passed`: Indicate that this action ran a test, and the test passed.\n\n### `intercept.so`\n\nSometimes, it's hard to know what a build tool's exact inputs and outputs will be ahead of time. For instance, a C++ compiler run will need to input all of the header files `#include`ed by the source file. There's no reasonable way to know what these might be in advance, much less look up the locations of files to satisfy each.\n\nTo solve this, Ekam implements a library which can be injected into a tool via `LD_PRELOAD` in order to intercept filesystem calls, automatically issues the proper Ekam commands to register inputs and outputs, and then map the call to the real disk path.\n\nThe interceptor library is implemented in `src/ekam/rules/intercept.c` and built by `src/ekam/rules/intercept.ekam-rule`. You may request it by requesting the tag `special:ekam-interceptor`, which maps to the compiled `intercept.so`, which you may then inject into an arbitrary command using `LD_PRELOAD`. See `src/ekam/rules/compile.ekam-rule` to see how this is done with the C++ compiler.\n\nWhen filesystem calls are being intercepted, an attempt to open a regular filename will be heuristically mapped to the closest file whose canonical name contains the full requested name as a suffix. For example, when processing `src/foo/bar/baz`, if the tool tries to open `qux/corge`, this could map to `src/foo/bar/qux/corge` or `src/qux/corge` or `src/grault/qux/corge`, but **not** `src/grault/corge` nor `src/qux/corge/grault` nor `src/corge`.\n\nIf you want to open a file by tag, the special virtual path `/ekam-provider//` can be used. For example, since the `include.ekam-rule` rule tags all C++ header files with `c++header:`, when we invoke the compiler using the inteceptor, we pass the flag `-I/ekam-provider/c++header`.\n\nIf you want to open a file purely by its whole canonical path (not using the heuristic that finds nearby files), you may do so by opening `/ekam-provider/canonical/`, since as described above every file gets tagged with `canonical:`.\n\n## Get Involved\n\nHave a question about Ekam, or want to contribute? Talk to us on the [Ekam discussion group](https://groups.google.com/group/ekam-tool).\n\nEkam is currently developed as part of the [Sandstorm.io](https://sandstorm.io) project and is primarily used by Sandstorm. If you like Ekam, consider [getting involved with Sandstorm](https://github.com/sandstorm-io/sandstorm/wiki/Get-Involved).\n" }, { "path": "qtcreator/COPYING.txt", "content": "Code in this directory is licensed under Apache 2.0 like normal.\n\nHowever, the icons under images/ are from the Qt Creator source code, and are thus\nsubject to Qt Creator's copyright and license. This probably means that the compiled\nplugin cannot legally be redistributed because it combines incompatible licenses\n(Apache vs. LGPL). I suppose I should either replace the icons or relicense the code.\nLet me know if anyone actually cares.\n" }, { "path": "qtcreator/EkamDashboard.json.in", "content": "{\n \\\"Name\\\" : \\\"EkamDashboard\\\",\n \\\"Version\\\" : \\\"0.0.1\\\",\n \\\"CompatVersion\\\" : \\\"0.0.1\\\",\n \\\"Vendor\\\" : \\\"KentonVarda\\\",\n \\\"Copyright\\\" : \\\"(C) Kenton Varda and Google Inc.\\\",\n \\\"License\\\" : \\\"Apache License v2.0\\\",\n \\\"Category\\\" : \\\"Qt Creator\\\",\n \\\"Description\\\" : \\\"Connects to the Ekam continuous build daemon and displays error markers.\\\",\n \\\"Url\\\" : \\\"http://ekam.googlecode.com\\\",\n $$dependencyList\n}\n" }, { "path": "qtcreator/EkamDashboard.pluginspec.in", "content": "\n KentonVarda\n (C) Google Inc.\n Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.\n Connects to the Ekam continuous build daemon and displays error markers.\n http://ekam.googlecode.com\n $$dependencyList\n\n\n" }, { "path": "qtcreator/ekamdashboard.pro", "content": "# Ekam Build System\n# Author: Kenton Varda (kenton@sandstorm.io)\n# Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nQMAKE_CXXFLAGS += -std=c++11 `pkg-config --cflags capnp`\nQMAKE_EXT_CPP += .c++\n\nTARGET = EkamDashboard\nTEMPLATE = lib\n\nDEFINES += EKAMDASHBOARD_LIBRARY\n\n# EkamDashboard files\n\nSOURCES += ekamdashboardplugin.cpp \\\n ekamtreewidget.cpp\n\nHEADERS += ekamdashboardplugin.h\\\n ekamdashboard_global.h\\\n ekamdashboardconstants.h \\\n ekamtreewidget.h\n\nCAPNPS += dashboard.capnp\n\nINCLUDEPATH += .\n\ncapnp_header.name = capnproto header\ncapnp_header.input = CAPNPS\ncapnp_header.output = ${QMAKE_FILE_BASE}.capnp.h\ncapnp_header.commands = \\\n capnp compile -oc++ --src-prefix=`dirname ${QMAKE_FILE_NAME}` ${QMAKE_FILE_NAME}; \\\n mv ${QMAKE_FILE_BASE}.capnp.c++ ${QMAKE_FILE_BASE}.capnp-noautolink.c++\ncapnp_header.variable_out = GENERATED_FILES\nQMAKE_EXTRA_COMPILERS += capnp_header\n\ncapnp_src.name = capnproto src\ncapnp_src.input = CAPNPS\ncapnp_src.output = ${QMAKE_FILE_BASE}.capnp-noautolink.c++\ncapnp_src.depends = ${QMAKE_FILE_BASE}.capnp.h\ncapnp_src.commands = true\ncapnp_src.variable_out = GENERATED_SOURCES\nQMAKE_EXTRA_COMPILERS += capnp_src\n\n# Qt Creator linking\n\n## set the QTC_SOURCE environment variable to override the setting here\nQTCREATOR_SOURCES = $$(QTC_SOURCE)\nisEmpty(QTCREATOR_SOURCES):error(\"Please set QTC_SOURCE to your Qt Creator source directory.\")\n\n## set the QTC_BUILD environment variable to override the setting here\nIDE_BUILD_TREE = $$(QTC_BUILD)\nisEmpty(IDE_BUILD_TREE):IDE_BUILD_TREE=$$(QTC_SOURCE)\n\n## uncomment to build plugin into user config directory\n## /plugins/\n## where is e.g.\n## \"%LOCALAPPDATA%\\Nokia\\qtcreator\" on Windows Vista and later\n## \"$XDG_DATA_HOME/Nokia/qtcreator\" or \"~/.local/share/data/Nokia/qtcreator\" on Linux\n## \"~/Library/Application Support/Nokia/Qt Creator\" on Mac\n# USE_USER_DESTDIR = yes\n\nPROVIDER = KentonVarda\n\n###### If the plugin can be depended upon by other plugins, this code needs to be outsourced to\n###### _dependencies.pri, where is the name of the directory containing the\n###### plugin's sources.\n\nQTC_PLUGIN_NAME = EkamDashboard\nQTC_LIB_DEPENDS += \\\n # nothing here at this time\n\nQTC_PLUGIN_DEPENDS += \\\n coreplugin \\\n projectexplorer \\\n cpptools \\\n texteditor\n\nQTC_PLUGIN_RECOMMENDS += \\\n # optional plugin dependencies. nothing here at this time\n\n###### End _dependencies.pri contents ######\n\nQT += network\n\ninclude($$QTCREATOR_SOURCES/src/qtcreatorplugin.pri)\n\nLIBS += -L$$IDE_PLUGIN_PATH/Nokia `pkg-config --libs capnp-rpc`\n\nRESOURCES += \\\n ekamdashboard.qrc\n\n" }, { "path": "qtcreator/ekamdashboard.qrc", "content": "\n \n images/state-blocked.png\n images/state-deleted.png\n images/state-done.png\n images/state-failed.png\n images/state-passed.png\n images/state-pending.png\n images/state-running.png\n images/dir.png\n images/dir-blocked.png\n images/dir-deleted.png\n images/dir-done.png\n images/dir-failed.png\n images/dir-passed.png\n images/dir-pending.png\n images/dir-running.png\n \n\n" }, { "path": "qtcreator/ekamdashboard_global.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef EKAMDASHBOARD_GLOBAL_H\n#define EKAMDASHBOARD_GLOBAL_H\n\n#include \n\n#if defined(EKAMDASHBOARD_LIBRARY)\n# define EKAMDASHBOARDSHARED_EXPORT Q_DECL_EXPORT\n#else\n# define EKAMDASHBOARDSHARED_EXPORT Q_DECL_IMPORT\n#endif\n\n#endif // EKAMDASHBOARD_GLOBAL_H\n\n" }, { "path": "qtcreator/ekamdashboardconstants.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef EKAMDASHBOARDCONSTANTS_H\n#define EKAMDASHBOARDCONSTANTS_H\n\nnamespace EkamDashboard {\nnamespace Constants {\n\nconst char ACTION_ID[] = \"EkamDashboard.Action\";\nconst char MENU_ID[] = \"EkamDashboard.Menu\";\nconst char TASK_CATEGORY_ID[] = \"EkamDashboard.Menu\";\n\n} // namespace EkamDashboard\n} // namespace Constants\n\n#endif // EKAMDASHBOARDCONSTANTS_H\n\n" }, { "path": "qtcreator/ekamdashboardplugin.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"ekamdashboardplugin.h\"\n#include \"ekamdashboardconstants.h\"\n#include \"ekamtreewidget.h\"\n\n#include \n#include \n#include \n\n#include \n\n#include \n\n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n\n#include \n\nnamespace EkamDashboard {\nnamespace Internal {\n\nQString toQString(kj::ArrayPtr str) {\n return QString::fromUtf8(str.begin(), str.size());\n}\n\nclass EkamDashboardPlugin::FakeAsyncInput final: public kj::AsyncInputStream {\npublic:\n kj::Promise read(void* buffer, size_t minBytes, size_t maxBytes) override {\n return tryRead(buffer, minBytes, maxBytes).then([=](size_t result) {\n KJ_REQUIRE(result >= minBytes, \"Premature EOF\") {\n // Pretend we read zeros from the input.\n memset(reinterpret_cast(buffer) + result, 0, minBytes - result);\n return minBytes;\n }\n return result;\n });\n }\n\n kj::Promise tryRead(void* buffer, size_t minBytes, size_t maxBytes) override {\n Request request(buffer, minBytes, maxBytes);\n\n if (byteQueue.size() > 0) {\n KJ_IF_MAYBE(size, request.consumeFrom(byteQueue)) {\n return *size;\n }\n }\n\n requests.push_back(kj::mv(request));\n return requests.back().finishLater();\n }\n\n void add(QByteArray bytes) {\n while (!requests.empty()) {\n auto& request = requests.front();\n\n KJ_IF_MAYBE(size, request.consumeFrom(bytes)) {\n request.fulfill(*size);\n requests.pop_front();\n } else {\n // Not enough bytes to satisfy the request.\n return;\n }\n }\n\n byteQueue.append(bytes);\n }\n\nprivate:\n class Request {\n public:\n Request(void* buffer, size_t minBytes, size_t maxBytes)\n : pos(reinterpret_cast(buffer)),\n minLeft(minBytes), maxLeft(maxBytes),\n alreadyRead(0) {}\n\n kj::Maybe consumeFrom(QByteArray& bytes) {\n size_t n = kj::min(maxLeft, bytes.size());\n memcpy(pos, bytes.data(), n);\n\n bytes.remove(0, n);\n\n if (n >= minLeft) {\n return alreadyRead + n;\n } else {\n pos += n;\n minLeft -= n;\n maxLeft -= n;\n alreadyRead += n;\n return nullptr;\n }\n }\n\n kj::Promise finishLater() {\n auto paf = kj::newPromiseAndFulfiller();\n fulfiller = kj::mv(paf.fulfiller);\n return kj::mv(paf.promise);\n }\n\n void fulfill(size_t amount) {\n fulfiller->fulfill(kj::mv(amount));\n }\n\n private:\n kj::byte* pos;\n size_t minLeft;\n size_t maxLeft;\n size_t alreadyRead;\n kj::Own> fulfiller;\n };\n\n std::deque requests;\n QByteArray byteQueue;\n};\n\nEkamDashboardPlugin::EkamDashboardPlugin()\n : hub(0), socket(0), seenHeader(false), waitScope(eventLoop),\n fakeInput(kj::heap()), readTask(nullptr) {}\n\nEkamDashboardPlugin::~EkamDashboardPlugin() noexcept {\n // Unregister objects from the plugin manager's object pool\n // Delete members\n}\n\nbool EkamDashboardPlugin::initialize(const QStringList &arguments, QString *errorString) {\n // Register objects in the plugin manager's object pool\n // Load settings\n // Add actions to menus\n // Connect to other plugins' signals\n // In the initialize method, a plugin can be sure that the plugins it\n // depends on have initialized their members.\n \n Q_UNUSED(arguments)\n Q_UNUSED(errorString)\n Core::ActionManager *am = Core::ActionManager::instance();\n\n QAction *action = new QAction(tr(\"EkamDashboard action\"), this);\n Core::Command *cmd = am->registerAction(action, Constants::ACTION_ID,\n Core::Context(Core::Constants::C_GLOBAL));\n cmd->setDefaultKeySequence(QKeySequence(tr(\"Ctrl+Alt+Meta+A\")));\n connect(action, SIGNAL(triggered()), this, SLOT(triggerAction()));\n\n Core::ActionContainer *menu = am->createMenu(Constants::MENU_ID);\n menu->menu()->setTitle(tr(\"EkamDashboard\"));\n menu->addAction(cmd);\n am->actionContainer(Core::Constants::M_TOOLS)->addMenu(menu);\n\n addAutoReleasedObject(new EkamTreeWidgetFactory(this));\n\n return true;\n}\n\nvoid EkamDashboardPlugin::extensionsInitialized() {\n // Retrieve objects from the plugin manager's object pool\n // In the extensionsInitialized method, a plugin can be sure that all\n // plugins that depend on it are completely initialized.\n\n ExtensionSystem::PluginManager *pm = ExtensionSystem::PluginManager::instance();\n hub = pm->getObject();\n\n hub->addCategory(Core::Id(Constants::TASK_CATEGORY_ID), QLatin1String(\"Ekam task\"));\n\n socket = new QTcpSocket(this);\n connect(socket, SIGNAL(error(QAbstractSocket::SocketError)),\n this, SLOT(socketError(QAbstractSocket::SocketError)));\n connect(socket, SIGNAL(readyRead()), this, SLOT(socketReady()));\n tryConnect();\n}\n\nExtensionSystem::IPlugin::ShutdownFlag EkamDashboardPlugin::aboutToShutdown() {\n // Save settings\n // Disconnect from signals that are not needed during shutdown\n // Hide UI (if you add UI that is not in the main window directly)\n\n delete socket;\n socket = 0;\n\n clearActions();\n\n return SynchronousShutdown;\n}\n\nvoid EkamDashboardPlugin::triggerAction() {\n// qDebug() << \"triggerAction() qdebug\";\n QMessageBox::information(Core::ICore::mainWindow(),\n tr(\"Action triggered\"),\n tr(\"This is an action from EkamDashboard.\"));\n\n hub->addTask(ProjectExplorer::Task(\n ProjectExplorer::Task::Error, QLatin1String(\"test error\"),\n Utils::FileName::fromUserInput(QLatin1String(\"/home/kenton/code/src/base/OwnedPtr.h\")), 10,\n Core::Id(Constants::TASK_CATEGORY_ID)));\n}\n\nvoid EkamDashboardPlugin::socketError(QAbstractSocket::SocketError error) {\n// qDebug() << \"Socket error: \" << error;\n reset();\n}\n\nvoid EkamDashboardPlugin::reset() {\n if (!resetting) {\n resetting = true;\n clearActions();\n QTimer::singleShot(5000, this, SLOT(retryConnection()));\n }\n}\n\nvoid EkamDashboardPlugin::retryConnection() {\n resetting = false;\n\n // Cancel any async parsing still happening.\n readTask = nullptr;\n\n // Reset the fake input to clear out its buffers.\n fakeInput = kj::heap();\n\n tryConnect();\n}\n\nvoid EkamDashboardPlugin::socketReady() {\n fakeInput->add(socket->readAll());\n eventLoop.run();\n}\n\nvoid EkamDashboardPlugin::tryConnect() {\n seenHeader = false;\n// qDebug() << \"Trying to connect...\";\n socket->connectToHost(QLatin1String(\"localhost\"), 41315);\n readTask = messageLoop().eagerlyEvaluate(\n [this](kj::Exception&& e){ KJ_LOG(ERROR, e); reset(); });\n}\n\nkj::Promise EkamDashboardPlugin::messageLoop() {\n return capnp::readMessage(*fakeInput).then([this](kj::Own&& message) {\n if (!seenHeader) {\n seenHeader = true;\n\n ekam::proto::Header::Reader header = message->getRoot();\n // qDebug() << \"Received header: \" << kj::str(header).cStr();\n projectRoot = toQString(header.getProjectRoot());\n } else {\n ekam::proto::TaskUpdate::Reader update = message->getRoot();\n // qDebug() << \"Received task update: \" << kj::str(update).cStr();\n\n ActionState*& slot = actions[update.getId()];\n if (slot == 0) {\n slot = new ActionState(this, update);\n } else {\n slot->applyUpdate(update);\n }\n\n if (slot->isDead()) {\n delete slot;\n actions.remove(update.getId());\n }\n }\n\n return messageLoop();\n });\n}\n\nQString EkamDashboardPlugin::findFile(const QString& canonicalPath) {\n QString srcpath = projectRoot + QLatin1String(\"/src/\") + canonicalPath;\n QString tmppath = projectRoot + QLatin1String(\"/tmp/\") + canonicalPath;\n if (QFile::exists(srcpath)) {\n return srcpath;\n } else if (QFile::exists(tmppath)) {\n return tmppath;\n } else {\n return canonicalPath;\n }\n}\n\nQList EkamDashboardPlugin::allActions() {\n QList result;\n\n foreach (ActionState* action, actions) {\n if (!action->isHidden()) {\n result << action;\n }\n }\n\n return result;\n}\n\nvoid EkamDashboardPlugin::clearActions() {\n foreach (ActionState* action, actions) {\n delete action;\n }\n actions.clear();\n}\n\n// =======================================================================================\n\nActionState::ActionState(\n EkamDashboardPlugin *plugin, ekam::proto::TaskUpdate::Reader initialUpdate)\n : plugin(plugin), state(initialUpdate.getState()),\n verb(toQString(initialUpdate.getVerb())),\n noun(toQString(initialUpdate.getNoun())),\n path(plugin->findFile(noun)),\n silent(initialUpdate.getSilent()) {\n auto log = initialUpdate.getLog();\n if (log != nullptr) {\n consumeLog(log);\n }\n if (!isHidden()) {\n plugin->unhideAction(this);\n }\n}\n\nActionState::~ActionState() noexcept {\n emit removed();\n clearTasks();\n}\n\nvoid ActionState::applyUpdate(ekam::proto::TaskUpdate::Reader update) {\n if (update.getState() != ekam::proto::TaskUpdate::State::UNCHANGED &&\n update.getState() != state) {\n bool wasHidden = isHidden();\n\n // Invalidate log when the task is deleted or it is re-running or scheduled to re-run.\n if (update.getState() == ekam::proto::TaskUpdate::State::PENDING ||\n update.getState() == ekam::proto::TaskUpdate::State::RUNNING ||\n update.getState() == ekam::proto::TaskUpdate::State::DELETED) {\n clearTasks();\n }\n\n state = update.getState();\n\n if (isHidden()) {\n if (wasHidden) {\n emit removed();\n }\n } else {\n if (wasHidden) {\n plugin->unhideAction(this);\n } else {\n emit stateChanged(state);\n }\n }\n }\n auto log = update.getLog();\n if (log != nullptr) {\n consumeLog(log);\n }\n if (state != ekam::proto::TaskUpdate::State::RUNNING && !leftoverLog.empty()) {\n parseLogLine(toQString(leftoverLog));\n leftoverLog.resize(0);\n }\n}\n\nvoid ActionState::clearTasks() {\n emit clearedTasks();\n foreach (const ProjectExplorer::Task& task, tasks) {\n plugin->taskHub()->removeTask(task);\n }\n tasks.clear();\n leftoverLog.resize(0);\n}\n\nvoid ActionState::consumeLog(kj::StringPtr log) {\n while (true) {\n KJ_IF_MAYBE(pos, log.findFirst('\\n')) {\n leftoverLog.addAll(log.begin(), log.begin() + *pos);\n log = log.slice(*pos + 1);\n parseLogLine(toQString(leftoverLog));\n leftoverLog.resize(0);\n } else {\n leftoverLog.addAll(log);\n return;\n }\n }\n}\n\nvoid ActionState::parseLogLine(QString line) {\n if (tasks.size() > 100) return; // avoid performance problems with too many tasks\n\n static const QRegExp FILE(QLatin1String(\"^([^ :]+):(.*)\"));\n static const QRegExp INDEX(QLatin1String(\"^([0-9]+):(.*)\"));\n static const QRegExp WARNING(QLatin1String(\"(.*[^a-zA-Z0-9])?warning:(.*)\"), Qt::CaseInsensitive);\n static const QRegExp ERROR(QLatin1String(\"(.*[^a-zA-Z0-9])?error:(.*)\"), Qt::CaseInsensitive);\n static const QRegExp FAILURE(QLatin1String(\" *failure *\"), Qt::CaseInsensitive);\n static const QRegExp FULL_LOG(QLatin1String(\"full log: tmp/(.*)\"));\n\n ProjectExplorer::Task::TaskType type = ProjectExplorer::Task::Unknown;\n QString file;\n int lineNo = -1;\n int columnNo = -1;\n\n // OMGWTF matching a QRegExp modifies the QRegExp object rather than returning some sort of match\n // object, so we must make copies. Hopefully using the copy constructor rather than constructing\n // directly from the pattern strings means they won't be re-compiled every time.\n QRegExp fullLog = FULL_LOG;\n QRegExp fileRe = FILE;\n if (fullLog.exactMatch(line)) {\n file = fullLog.capturedTexts()[1];\n file = plugin->findFile(file);\n if (!file.startsWith(QLatin1Char('/'))) {\n file.clear();\n }\n } else if (fileRe.exactMatch(line)) {\n file = fileRe.capturedTexts()[1];\n if (file.startsWith(QLatin1String(\"/ekam-provider/c++header/\"))) {\n file.remove(0, strlen(\"/ekam-provider/c++header/\"));\n } else if (file.startsWith(QLatin1String(\"/ekam-provider/canonical/\"))) {\n file.remove(0, strlen(\"/ekam-provider/canonical/\"));\n }\n file = plugin->findFile(file);\n if (file.startsWith(QLatin1Char('/'))) {\n line = fileRe.capturedTexts()[2];\n } else {\n // Failed to find the file on disk. Maybe it's not actually a file. Leave it in the error\n // text.\n file.clear();\n }\n }\n\n QRegExp indexRe = INDEX;\n if (indexRe.exactMatch(line)) {\n type = ProjectExplorer::Task::Unknown;\n lineNo = indexRe.capturedTexts()[1].toInt();\n line = indexRe.capturedTexts()[2];\n if (indexRe.exactMatch(line)) {\n columnNo = indexRe.capturedTexts()[1].toInt();\n line = indexRe.capturedTexts()[2];\n }\n }\n\n QRegExp errorRe = ERROR;\n QRegExp warningRe = WARNING;\n QRegExp failureRe = FAILURE;\n if (errorRe.exactMatch(line)) {\n type = ProjectExplorer::Task::Error;\n } else if (warningRe.exactMatch(line)) {\n type = ProjectExplorer::Task::Warning;\n } else if (failureRe.exactMatch(line)) {\n type = ProjectExplorer::Task::Error;\n }\n\n // Qt Creator tasks don't support column numbers, so add it back into the error text.\n if (columnNo != -1) {\n line = QLatin1String(\"(col \") + QString::number(columnNo) + QLatin1String(\") \") + line;\n }\n\n tasks.append(ProjectExplorer::Task(type, line, Utils::FileName::fromUserInput(file), lineNo,\n Core::Id(Constants::TASK_CATEGORY_ID)));\n plugin->taskHub()->addTask(tasks.back());\n emit addedTask(tasks.back());\n}\n\n} // namespace Internal\n} // namespace EkamDashboard\n\n" }, { "path": "qtcreator/ekamdashboardplugin.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef EKAMDASHBOARD_H\n#define EKAMDASHBOARD_H\n\n#include \"ekamdashboard_global.h\"\n\n#include \n\n#include \n#include \n#include \n#include \n#include \n#include \n#include \n\n#include \n\n#include \n#include \n#include \n#include \n\n#include \n#include \n#include \"dashboard.capnp.h\"\n\nnamespace ProjectExplorer {\nclass TaskHub;\n}\n\nnamespace EkamDashboard {\nnamespace Internal {\n\nclass EkamDashboardPlugin;\n\nclass ActionState: public QObject {\n Q_OBJECT\n\npublic:\n ActionState(EkamDashboardPlugin* plugin, ekam::proto::TaskUpdate::Reader initialUpdate);\n ~ActionState() noexcept;\n\n // Returns true if the action went from silent to non-silent, and thus a newAction event should\n // be fired.\n void applyUpdate(ekam::proto::TaskUpdate::Reader update);\n\n bool isDead() {\n return state == ekam::proto::TaskUpdate::State::DELETED;\n }\n\n ekam::proto::TaskUpdate::State getState() { return state; }\n const QString& getVerb() { return verb; }\n const QString& getNoun() { return noun; }\n const QString& getPath() { return path; }\n bool isHidden() {\n return silent && state != ekam::proto::TaskUpdate::State::FAILED;\n }\n ProjectExplorer::Task* firstTask() { return tasks.empty() ? 0 : &tasks.first(); }\n\nsignals:\n void removed();\n void stateChanged(ekam::proto::TaskUpdate::State state);\n void clearedTasks();\n void addedTask(const ProjectExplorer::Task& task);\n\nprivate:\n EkamDashboardPlugin* plugin;\n ekam::proto::TaskUpdate::State state;\n QString verb;\n QString noun;\n QString path;\n bool silent;\n kj::Vector leftoverLog;\n QList tasks;\n\n void clearTasks();\n void consumeLog(kj::StringPtr log);\n void parseLogLine(QString line);\n};\n\nclass EkamDashboardPlugin : public ExtensionSystem::IPlugin {\n Q_OBJECT\n Q_PLUGIN_METADATA(IID \"org.qt-project.Qt.QtCreatorPlugin\" FILE \"EkamDashboard.json\")\n \npublic:\n EkamDashboardPlugin();\n ~EkamDashboardPlugin() noexcept;\n\n bool initialize(const QStringList &arguments, QString *errorString);\n void extensionsInitialized();\n ShutdownFlag aboutToShutdown();\n\n ProjectExplorer::TaskHub* taskHub() { return hub; }\n QString findFile(const QString& canonicalPath);\n\n QList allActions();\n\n void unhideAction(ActionState* action) {\n emit newAction(action);\n }\n\nsignals:\n void newAction(ActionState* action);\n\nprivate slots:\n void triggerAction();\n void socketError(QAbstractSocket::SocketError);\n void retryConnection();\n void socketReady();\n\nprivate:\n class FakeAsyncInput;\n\n ProjectExplorer::TaskHub* hub;\n QTcpSocket* socket;\n bool seenHeader;\n QString projectRoot;\n QHash actions;\n kj::EventLoop eventLoop;\n kj::WaitScope waitScope;\n kj::Own fakeInput;\n kj::Promise readTask;\n bool resetting = false;\n\n void reset();\n void tryConnect();\n kj::Promise messageLoop();\n void clearActions();\n};\n\n} // namespace Internal\n} // namespace EkamDashboard\n\n#endif // EKAMDASHBOARD_H\n\n" }, { "path": "qtcreator/ekamtreewidget.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"ekamtreewidget.h\"\n#include \"ekamdashboardplugin.h\"\n\n#include \n\n#include \n#include \n#include \n\nnamespace EkamDashboard {\nnamespace Internal {\n\nstatic constexpr ekam::proto::TaskUpdate::State DEFAULT_STATE =\n ekam::proto::TaskUpdate::State::UNCHANGED;\n\n// This defines the priority ordering of states. The state (and icon) for a directory will be\n// chosen based on the highest-priority state of its children.\nstatic const ekam::proto::TaskUpdate::State ORDERED_STATES[] = {\n DEFAULT_STATE,\n ekam::proto::TaskUpdate::State::DELETED,\n ekam::proto::TaskUpdate::State::DONE,\n ekam::proto::TaskUpdate::State::PASSED,\n ekam::proto::TaskUpdate::State::FAILED,\n ekam::proto::TaskUpdate::State::PENDING,\n ekam::proto::TaskUpdate::State::BLOCKED,\n ekam::proto::TaskUpdate::State::RUNNING,\n};\n\nstatic int STATE_PRIORITIES[16];\n\nstruct StatePrioritiesInitializer {\n StatePrioritiesInitializer() {\n for (size_t i = 0; i < (sizeof(STATE_PRIORITIES) / sizeof(STATE_PRIORITIES[0])); i++) {\n STATE_PRIORITIES[i] = -1;\n }\n for (size_t i = 0; i < (sizeof(ORDERED_STATES) / sizeof(ORDERED_STATES[0])); i++) {\n STATE_PRIORITIES[static_cast(ORDERED_STATES[i])] = i;\n }\n }\n};\nstatic StatePrioritiesInitializer statePrioritiesInitializer;\n\n// =======================================================================================\n\nEkamTreeNode::EkamTreeNode(EkamTreeModel* tree)\n : QObject(tree), tree(tree), isDirectory(true), parentNode(0), action(0), state(DEFAULT_STATE) {}\nEkamTreeNode::EkamTreeNode(EkamTreeNode* parent, const QString& name, bool isDirectory)\n : QObject(parent), tree(parent->tree), isDirectory(isDirectory), name(name),\n parentNode(parent), action(0), state(DEFAULT_STATE) {}\nEkamTreeNode::~EkamTreeNode() {}\n\nint EkamTreeNode::row() {\n if (this == tree->root) {\n return -1;\n }\n\n int result = parentNode->childNodes.indexOf(this);\n if (result == -1) {\n qWarning() << \"parentNode->childNodes doesn't contain this?\";\n }\n return result;\n}\n\nQModelIndex EkamTreeNode::index() {\n if (this == tree->root) {\n return QModelIndex();\n }\n\n int rowNum = row();\n if (rowNum == -1) {\n return QModelIndex();\n }\n\n return tree->createIndex(rowNum, 0, this);\n}\n\nvoid EkamTreeNode::actionStateChanged(ekam::proto::TaskUpdate::State newState) {\n stateChanged(newState);\n}\n\nvoid EkamTreeNode::actionRemoved() {\n setAction(0);\n if (parentNode != 0) {\n parentNode->removeChild(this);\n }\n// QModelIndex myIndex = index();\n// emit tree->dataChanged(myIndex, myIndex);\n}\n\nvoid EkamTreeNode::createNode(const QString& noun, const QString& verb, ActionState* action) {\n int slash = noun.indexOf(QLatin1Char('/'));\n\n QString childName = (slash == -1) ? QString(QLatin1String(\"%1 (%2)\")).arg(noun, verb) : noun.mid(0, slash);\n\n QList::iterator iter = childNodes.begin();\n while (iter < childNodes.end() && (*iter)->name < childName) {\n ++iter;\n }\n\n EkamTreeNode* selectedNode;\n bool isNew = iter == childNodes.end() || (*iter)->name != childName;\n\n if (isNew) {\n QModelIndex i = index();\n int r = iter - childNodes.begin();\n tree->beginInsertRows(i, r, r);\n selectedNode = new EkamTreeNode(this, childName, slash != -1);\n childNodes.insert(iter, selectedNode);\n } else {\n selectedNode = *iter;\n }\n\n if (slash == -1) {\n selectedNode->setAction(action);\n } else {\n selectedNode->createNode(noun.right(noun.size() - slash - 1), verb, action);\n }\n\n if (isNew) {\n tree->endInsertRows();\n } else {\n QModelIndex nodeIndex = selectedNode->index();\n emit tree->dataChanged(nodeIndex, nodeIndex);\n }\n}\n\nQVariant EkamTreeNode::data(int role) {\n switch (role) {\n case Qt::DisplayRole:\n case Qt::EditRole:\n // string title\n return name;\n\n case Qt::ToolTipRole:\n // string tooltip\n if (action == 0) {\n return QVariant();\n } else {\n action->getPath();\n }\n\n case Qt::DecorationRole:\n // icon\n if (isDirectory) {\n switch (state) {\n case ekam::proto::TaskUpdate::State::DELETED:\n return QIcon(QLatin1String(\":/ekamdashboard/images/dir-deleted.png\"));\n case ekam::proto::TaskUpdate::State::PENDING:\n return QIcon(QLatin1String(\":/ekamdashboard/images/dir-pending.png\"));\n case ekam::proto::TaskUpdate::State::RUNNING:\n return QIcon(QLatin1String(\":/ekamdashboard/images/dir-running.png\"));\n case ekam::proto::TaskUpdate::State::DONE:\n return QIcon(QLatin1String(\":/ekamdashboard/images/dir-done.png\"));\n case ekam::proto::TaskUpdate::State::PASSED:\n return QIcon(QLatin1String(\":/ekamdashboard/images/dir-passed.png\"));\n case ekam::proto::TaskUpdate::State::FAILED:\n return QIcon(QLatin1String(\":/ekamdashboard/images/dir-failed.png\"));\n case ekam::proto::TaskUpdate::State::BLOCKED:\n // Use pending icon for blocked.\n return QIcon(QLatin1String(\":/ekamdashboard/images/dir-pending.png\"));\n case DEFAULT_STATE:\n return QIcon(QLatin1String(\":/ekamdashboard/images/dir.png\"));\n }\n } else {\n switch (state) {\n case DEFAULT_STATE:\n case ekam::proto::TaskUpdate::State::DELETED:\n return QIcon(QLatin1String(\":/ekamdashboard/images/state-deleted.png\"));\n case ekam::proto::TaskUpdate::State::PENDING:\n return QIcon(QLatin1String(\":/ekamdashboard/images/state-pending.png\"));\n case ekam::proto::TaskUpdate::State::RUNNING:\n return QIcon(QLatin1String(\":/ekamdashboard/images/state-running.png\"));\n case ekam::proto::TaskUpdate::State::DONE:\n return QIcon(QLatin1String(\":/ekamdashboard/images/state-done.png\"));\n case ekam::proto::TaskUpdate::State::PASSED:\n return QIcon(QLatin1String(\":/ekamdashboard/images/state-passed.png\"));\n case ekam::proto::TaskUpdate::State::FAILED:\n return QIcon(QLatin1String(\":/ekamdashboard/images/state-failed.png\"));\n case ekam::proto::TaskUpdate::State::BLOCKED:\n // Use pending icon for blocked.\n return QIcon(QLatin1String(\":/ekamdashboard/images/state-pending.png\"));\n }\n }\n qWarning() << \"Can't get here.\";\n break;\n\n case Qt::FontRole:\n // We could bold this or something.\n QFont result;\n if (isDirectory) {\n result.setBold(true);\n }\n return result;\n }\n\n return QVariant();\n}\n\nvoid EkamTreeNode::setAction(ActionState* newAction) {\n if (action != 0) {\n disconnect(action, SIGNAL(stateChanged(ekam::proto::TaskUpdate::State)),\n this, SLOT(actionStateChanged(ekam::proto::TaskUpdate::State)));\n disconnect(action, SIGNAL(removed()), this, SLOT(actionRemoved()));\n }\n\n action = newAction;\n stateChanged(action == 0 ? DEFAULT_STATE : action->getState());\n\n if (action != 0) {\n connect(action, SIGNAL(stateChanged(ekam::proto::TaskUpdate::State)),\n this, SLOT(actionStateChanged(ekam::proto::TaskUpdate::State)));\n connect(action, SIGNAL(removed()), this, SLOT(actionRemoved()));\n }\n}\n\nvoid EkamTreeNode::stateChanged(ekam::proto::TaskUpdate::State newState) {\n if (state != newState) {\n state = newState;\n QModelIndex myIndex = index();\n emit tree->dataChanged(myIndex, myIndex);\n if (parentNode != 0) {\n parentNode->childStateChanged();\n }\n }\n}\n\nvoid EkamTreeNode::childStateChanged() {\n ekam::proto::TaskUpdate::State maxState = DEFAULT_STATE;\n int maxStatePriority = STATE_PRIORITIES[static_cast(maxState)];\n\n foreach (EkamTreeNode* child, childNodes) {\n int childPriority = STATE_PRIORITIES[static_cast(child->state)];\n if (childPriority > maxStatePriority) {\n maxState = child->state;\n maxStatePriority = childPriority;\n }\n }\n\n stateChanged(maxState);\n}\n\nvoid EkamTreeNode::removeChild(EkamTreeNode* child) {\n int r = child->row();\n tree->beginRemoveRows(index(), r, r);\n childNodes.erase(childNodes.begin() + r);\n tree->endRemoveRows();\n if (childNodes.empty() && parentNode != 0) {\n parentNode->removeChild(this);\n }\n}\n\n// =======================================================================================\n\nEkamTreeModel::EkamTreeModel(EkamDashboardPlugin* plugin, QObject* parent)\n : QAbstractItemModel(parent), plugin(plugin), root(new EkamTreeNode(this)) {\n// qDebug() << \"EkamTreeModel::EkamTreeModel(...)\";\n\n connect(plugin, SIGNAL(newAction(ActionState*)), this, SLOT(newAction(ActionState*)));\n\n foreach (ActionState* action, plugin->allActions()) {\n newAction(action);\n }\n}\n\nEkamTreeModel::~EkamTreeModel() {}\n\nQModelIndex EkamTreeModel::index(int row, int column, const QModelIndex & parent) const {\n// qDebug() << \"EkamTreeModel::index(\" << row << \", \" << column << \", \" << parent << \")\";\n\n if (column != 0) {\n // Invalid.\n return QModelIndex();\n }\n\n EkamTreeNode* parentNode = indexToNode(parent);\n if (row < 0 || row >= parentNode->childCount()) {\n // Out of bounds.\n return QModelIndex();\n }\n\n return createIndex(row, 0, parentNode->getChild(row));\n}\n\nQModelIndex EkamTreeModel::parent(const QModelIndex &index) const {\n// qDebug() << \"EkamTreeModel::parent(\" << index << \")\";\n\n EkamTreeNode* node = indexToNode(index);\n if (node == root) {\n // This should never happen?\n qWarning() << \"Called parent() on invisible root object?\";\n return QModelIndex();\n } else {\n return reinterpret_cast(node->parent())->index();\n }\n}\n\nQVariant EkamTreeModel::data(const QModelIndex &index, int role) const {\n// qDebug() << \"EkamTreeModel::data(\" << index << \", \" << role << \") = \"\n// << indexToNode(index)->data(role);\n return indexToNode(index)->data(role);\n}\n\nint EkamTreeModel::rowCount(const QModelIndex & parent) const {\n// qDebug() << \"EkamTreeModel::rowCount(\" << parent << \") = \"\n// << indexToNode(parent)->childCount();\n return indexToNode(parent)->childCount();\n}\n\nint EkamTreeModel::columnCount(const QModelIndex & parent) const {\n// qDebug() << \"EkamTreeModel::columnCount(\" << parent << \") = 1\";\n Q_UNUSED(parent);\n return 1;\n}\n\nbool EkamTreeModel::hasChildren(const QModelIndex & parent) const {\n// qDebug() << \"EkamTreeModel::hasChildren(\" << parent << \") = \"\n// << (indexToNode(parent)->childCount() > 0);\n return indexToNode(parent)->childCount() > 0;\n}\n\nvoid EkamTreeModel::newAction(ActionState* action) {\n// qDebug() << \"EkamTreeModel::newAction(\" << action->getVerb() << \":\" << action->getNoun() << \")\";\n root->createNode(action->getNoun(), action->getVerb(), action);\n}\n\n// =======================================================================================\n\nEkamTreeWidget::EkamTreeWidget(EkamDashboardPlugin* plugin)\n : QWidget(), plugin(plugin) {\n model = new EkamTreeModel(plugin, this);\n\n view = new Utils::NavigationTreeView(this);\n view->setModel(model);\n setFocusProxy(view);\n\n QVBoxLayout *layout = new QVBoxLayout();\n layout->addWidget(view);\n layout->setContentsMargins(0, 0, 0, 0);\n setLayout(layout);\n\n connect(view, SIGNAL(doubleClicked(QModelIndex)), this, SLOT(jumpTo(QModelIndex)));\n}\n\nEkamTreeWidget::~EkamTreeWidget() {}\n\nvoid EkamTreeWidget::jumpTo(const QModelIndex& index) {\n ActionState* action = model->indexToNode(index)->getAction();\n if (action != 0) {\n const ProjectExplorer::Task* task = action->firstTask();\n Core::EditorManager* editorManager = Core::EditorManager::instance();\n if (task == 0) {\n // Open in editor.\n editorManager->openEditor(action->getPath());\n } else {\n // Open first task in editor and issues list.\n plugin->taskHub()->taskMarkClicked(task->taskId);\n QString name = task->file.toString();\n if (name.isEmpty()) {\n name = action->getPath();\n }\n editorManager->openEditorAt(name, task->line);\n }\n }\n}\n\n// =======================================================================================\n\nEkamTreeWidgetFactory::EkamTreeWidgetFactory(EkamDashboardPlugin* plugin)\n : INavigationWidgetFactory(), plugin(plugin) {\n setId(\"EkamActions\");\n setDisplayName(QLatin1String(\"Ekam Actions\"));\n setPriority(100);\n}\nEkamTreeWidgetFactory::~EkamTreeWidgetFactory() {}\n\nCore::NavigationView EkamTreeWidgetFactory::createWidget() {\n Core::NavigationView result;\n\n EkamTreeWidget* tree = new EkamTreeWidget(plugin);\n result.widget = tree;\n\n return result;\n}\n\n} // namespace Internal\n} // namespace EkamDashboard\n" }, { "path": "qtcreator/ekamtreewidget.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef EKAMTREEWIDGET_H\n#define EKAMTREEWIDGET_H\n\n#include \n\n#include \n#include \n#include \n\n#include \n\n#include \"dashboard.capnp.h\"\n\nnamespace EkamDashboard {\nnamespace Internal {\n\nclass EkamDashboardPlugin;\nclass EkamTreeModel;\nclass ActionState;\n\nclass EkamTreeNode : public QObject {\n Q_OBJECT\npublic:\n explicit EkamTreeNode(EkamTreeModel* tree);\n EkamTreeNode(EkamTreeNode* parent, const QString& name, bool isDirectory);\n virtual ~EkamTreeNode();\n\n int row();\n QModelIndex index();\n\n void createNode(const QString& noun, const QString& verb, ActionState* action);\n\n QVariant data(int role);\n\n int childCount() {\n return childNodes.size();\n }\n EkamTreeNode* getChild(int index) {\n return childNodes.at(index);\n }\n\n ActionState* getAction() {\n return action;\n }\n\nprivate slots:\n void actionStateChanged(ekam::proto::TaskUpdate::State newState);\n void actionRemoved();\n\nprivate:\n EkamTreeModel* tree;\n\n bool isDirectory;\n QString name;\n\n EkamTreeNode* parentNode;\n QList childNodes;\n\n ActionState* action;\n ekam::proto::TaskUpdate::State state;\n\n void setAction(ActionState* newAction);\n void stateChanged(ekam::proto::TaskUpdate::State newState);\n void childStateChanged();\n void removeChild(EkamTreeNode* child);\n};\n\nclass EkamTreeModel : public QAbstractItemModel {\n Q_OBJECT\npublic:\n EkamTreeModel(EkamDashboardPlugin* plugin, QObject* parent = 0);\n virtual ~EkamTreeModel();\n\n virtual QModelIndex index(int row, int column, const QModelIndex & parent = QModelIndex()) const;\n virtual QModelIndex parent(const QModelIndex &index) const;\n virtual QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const;\n\n virtual int rowCount(const QModelIndex & parent = QModelIndex()) const;\n virtual int columnCount(const QModelIndex & parent = QModelIndex()) const;\n virtual bool hasChildren(const QModelIndex & parent = QModelIndex()) const;\n\n EkamTreeNode* indexToNode(const QModelIndex& index) const {\n return index.isValid() ? reinterpret_cast(index.internalPointer()) : root;\n }\n\nprivate slots:\n void newAction(ActionState* action);\n\nprivate:\n friend class EkamTreeNode;\n\n EkamDashboardPlugin* plugin;\n EkamTreeNode* root;\n};\n\nclass EkamTreeWidget : public QWidget {\n Q_OBJECT\npublic:\n explicit EkamTreeWidget(EkamDashboardPlugin *plugin = 0);\n virtual ~EkamTreeWidget();\n\nprivate slots:\n void jumpTo(const QModelIndex& index);\n\nprivate:\n EkamDashboardPlugin* plugin;\n\n EkamTreeModel* model;\n QTreeView* view;\n};\n\nclass EkamTreeWidgetFactory : public Core::INavigationWidgetFactory {\n Q_OBJECT\n\npublic:\n explicit EkamTreeWidgetFactory(EkamDashboardPlugin* plugin);\n virtual ~EkamTreeWidgetFactory();\n\n virtual Core::NavigationView createWidget();\n\nprivate:\n EkamDashboardPlugin* plugin;\n};\n\n} // namespace Internal\n} // namespace EkamDashboard\n\n#endif // EKAMTREEWIDGET_H\n" }, { "path": "src/base/Debug.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"Debug.h\"\n\n#include \n\nnamespace ekam {\n\nDebugMessage::Severity DebugMessage::logLevel = WARNING;\nint DebugMessage::counter = 0;\n\nstatic const char* SEVERITY_NAMES[] = {\n \"INFO\", \"WARNING\", \"ERROR\"\n};\n\nDebugMessage::DebugMessage(Severity severity, const char* filename, int line) {\n *this << \"ekam debug: \" << SEVERITY_NAMES[severity] << \": \"\n << filename << \":\" << line << \": \";\n}\nDebugMessage::~DebugMessage() {\n // TODO: We really need to buffer the message and write it all at once to avoid interleaved\n // text when multiprocessing.\n fputs(\"\\n\", stderr);\n fflush(stderr);\n ++counter;\n}\n\nDebugMessage& DebugMessage::operator<<(const char* value) {\n fputs(value, stderr);\n return *this;\n}\n\nDebugMessage& DebugMessage::operator<<(const std::string& value) {\n fwrite(value.data(), sizeof(char), value.size(), stderr);\n return *this;\n}\n\n#define HANDLE_TYPE(TYPE, FORMAT) \\\nDebugMessage& DebugMessage::operator<<(TYPE value) { \\\n fprintf(stderr, FORMAT, value); \\\n return *this; \\\n}\n\nHANDLE_TYPE(char, \"%c\");\nHANDLE_TYPE(signed char, \"%hhd\");\nHANDLE_TYPE(unsigned char, \"%hhu\");\nHANDLE_TYPE(short, \"%hd\");\nHANDLE_TYPE(unsigned short, \"%hu\");\nHANDLE_TYPE(int, \"%d\");\nHANDLE_TYPE(unsigned int, \"%u\");\nHANDLE_TYPE(long, \"%ld\");\nHANDLE_TYPE(unsigned long, \"%lu\");\nHANDLE_TYPE(long long, \"%lld\");\nHANDLE_TYPE(unsigned long long, \"%llu\");\nHANDLE_TYPE(float, \"%g\");\nHANDLE_TYPE(double, \"%g\");\nHANDLE_TYPE(const void*, \"%p\");\n\n} // namespace ekam\n" }, { "path": "src/base/Debug.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_BASE_DEBUGLOG_H_\n#define KENTONSCODE_BASE_DEBUGLOG_H_\n\n#include \n\nnamespace ekam {\n\nclass DebugMessage {\npublic:\n enum Severity {\n INFO,\n WARNING,\n ERROR\n };\n\n DebugMessage(Severity severity, const char* filename, int line);\n ~DebugMessage();\n\n DebugMessage& operator<<(const char* value);\n DebugMessage& operator<<(const std::string& value);\n DebugMessage& operator<<(char value);\n DebugMessage& operator<<(signed char value);\n DebugMessage& operator<<(unsigned char value);\n DebugMessage& operator<<(short value);\n DebugMessage& operator<<(unsigned short value);\n DebugMessage& operator<<(int value);\n DebugMessage& operator<<(unsigned int value);\n DebugMessage& operator<<(long value);\n DebugMessage& operator<<(unsigned long value);\n DebugMessage& operator<<(long long value);\n DebugMessage& operator<<(unsigned long long value);\n DebugMessage& operator<<(float value);\n DebugMessage& operator<<(double value);\n DebugMessage& operator<<(const void* ptr);\n\n inline static bool shouldLog(Severity severity, const char*, int) {\n return severity >= logLevel;\n }\n\n inline static void setLogLevel(Severity severity) {\n logLevel = severity;\n }\n\n // Useful for detecting if any log messages have been printed, e.g. to avoid clobbering them\n // with terminal manipulations.\n inline static int getMessageCount() { return counter; }\n\nprivate:\n static Severity logLevel;\n static int counter;\n};\n\n#define DEBUG_LOG(SEVERITY) \\\n if (!::ekam::DebugMessage::shouldLog(::ekam::DebugMessage::SEVERITY, __FILE__, __LINE__)) {} \\\n else ::ekam::DebugMessage(::ekam::DebugMessage::SEVERITY, __FILE__, __LINE__)\n\n#define DEBUG_INFO DEBUG_LOG(INFO)\n#define DEBUG_WARNING DEBUG_LOG(WARNING)\n#define DEBUG_ERROR DEBUG_LOG(ERROR)\n\n} // namespace ekam\n\n#endif // KENTONSCODE_BASE_DEBUGLOG_H_\n" }, { "path": "src/base/Hash.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"Hash.h\"\n\n#include \n\n#include \"sha256.h\"\n\nnamespace ekam {\n\nnamespace {\n\nchar HexDigit(unsigned int value) {\n value &= 0x0F;\n if (value < 10) {\n return '0' + value;\n } else {\n return 'a' + value - 10;\n }\n}\n\n} // anonymous namespace\n\nHash Hash::of(const std::string& data) {\n return Builder().add(data).build();\n}\n\nHash Hash::of(void* data, size_t size) {\n return Builder().add(data, size).build();\n}\n\n// Note: Since this is in static space it will be automatically initialized to zero.\nconst Hash Hash::NULL_HASH;\n\nstd::string Hash::toString() const {\n std::string result;\n result.reserve(sizeof(hash) * 2);\n for (unsigned int i = 0; i < sizeof(hash); i++) {\n result.push_back(HexDigit(hash[i] >> 4));\n result.push_back(HexDigit(hash[i]));\n }\n return result;\n}\n\nHash::Builder::Builder() {\n SHA256_Init(&context);\n}\n\nHash::Builder& Hash::Builder::add(const std::string& data) {\n SHA256_Update(&context, data.data(), data.size());\n return *this;\n}\n\nHash::Builder& Hash::Builder::add(void* data, size_t size) {\n SHA256_Update(&context, data, size);\n return *this;\n}\n\nHash Hash::Builder::build() {\n Hash result;\n SHA256_Final(result.hash, &context);\n return result;\n}\n\n} // namespace ekam\n" }, { "path": "src/base/Hash.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_BASE_HASH_H_\n#define KENTONSCODE_BASE_HASH_H_\n\n#include \n#include \n#include \n\n#include \"sha256.h\"\n\nnamespace ekam {\n\nclass Hash {\npublic:\n inline Hash() {}\n\n class Builder {\n public:\n Builder();\n Builder& add(const std::string& data);\n Builder& add(void* data, size_t size);\n Hash build();\n\n private:\n SHA256Context context;\n };\n\n static Hash of(const std::string& data);\n static Hash of(void* data, size_t size);\n static const Hash NULL_HASH;\n\n std::string toString() const;\n\n inline bool operator==(const Hash& other) const {\n return memcmp(hash, other.hash, sizeof(hash)) == 0;\n }\n inline bool operator!=(const Hash& other) const {\n return memcmp(hash, other.hash, sizeof(hash)) != 0;\n }\n inline bool operator<(const Hash& other) const {\n return memcmp(hash, other.hash, sizeof(hash)) < 0;\n }\n inline bool operator>(const Hash& other) const {\n return memcmp(hash, other.hash, sizeof(hash)) > 0;\n }\n inline bool operator<=(const Hash& other) const {\n return memcmp(hash, other.hash, sizeof(hash)) <= 0;\n }\n inline bool operator>=(const Hash& other) const {\n return memcmp(hash, other.hash, sizeof(hash)) >= 0;\n }\n\n class StlHashFunc {\n public:\n inline size_t operator()(const Hash& h) const {\n return h.shortHash;\n }\n };\n\nprivate:\n union {\n unsigned char hash[32];\n size_t shortHash;\n };\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_BASE_HASH_H_\n" }, { "path": "src/base/OwnedPtr.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"OwnedPtr.h\"\n\nnamespace ekam {\n\n} // namespace ekam\n" }, { "path": "src/base/OwnedPtr.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_BASE_OWNEDPTR_H_\n#define KENTONSCODE_BASE_OWNEDPTR_H_\n\n#include \n#include \n#include \n#include \n#include \n#include \n#include \n\n#ifdef __CDT_PARSER__\n#define noexcept\n#define constexpr\nnamespace std { struct nullptr_t; }\n#endif\n\nnamespace ekam {\n\ntemplate \ninline void deleteEnsuringCompleteType(T* ptr) {\n enum { type_must_be_complete = sizeof(T) };\n delete ptr;\n}\n\ntemplate \nclass OwnedPtr {\npublic:\n OwnedPtr() : ptr(NULL) {}\n OwnedPtr(const OwnedPtr&) = delete;\n OwnedPtr(OwnedPtr&& other) : ptr(other.releaseRaw()) {}\n template \n OwnedPtr(OwnedPtr&& other) : ptr(other.releaseRaw()) {}\n OwnedPtr(std::nullptr_t) : ptr(NULL) {}\n ~OwnedPtr() {\n deleteEnsuringCompleteType(ptr);\n }\n\n OwnedPtr& operator=(const OwnedPtr&) = delete;\n OwnedPtr& operator=(OwnedPtr&& other) {\n reset(other.releaseRaw());\n return *this;\n }\n\n template \n OwnedPtr& operator=(OwnedPtr&& other) {\n reset(other.releaseRaw());\n return *this;\n }\n\n T* get() const { return ptr; }\n T* operator->() const { assert(ptr != NULL); return ptr; }\n T& operator*() const { assert(ptr != NULL); return *ptr; }\n\n OwnedPtr release() {\n return OwnedPtr(releaseRaw());\n }\n\n void clear() {\n reset(NULL);\n }\n\n bool operator==(const T* other) { return ptr == other; }\n bool operator!=(const T* other) { return ptr != other; }\n\nprivate:\n T* ptr;\n\n explicit OwnedPtr(T* ptr) : ptr(ptr) {}\n\n void reset(T* newValue) {\n T* oldValue = ptr;\n ptr = newValue;\n deleteEnsuringCompleteType(oldValue);\n }\n\n T* releaseRaw() {\n T* result = ptr;\n ptr = NULL;\n return result;\n }\n\n template \n friend class OwnedPtr;\n template \n friend OwnedPtr newOwned(Params&&... params);\n template \n friend class SmartPtr;\n template \n friend class OwnedPtrVector;\n template \n friend class OwnedPtrDeque;\n template \n friend class OwnedPtrQueue;\n template \n friend class OwnedPtrMap;\n};\n\ntemplate \nOwnedPtr newOwned(Params&&... params) {\n return OwnedPtr(new T(std::forward(params)...));\n}\n\ntemplate \nclass Indirect {\npublic:\n template \n Indirect(Params&&... params): ptr(newOwned(std::forward(params)...)) {}\n Indirect(Indirect&& other): ptr(other.ptr.release()) {}\n Indirect(const Indirect& other): ptr(newOwned(*other.ptr)) {}\n\n Indirect& operator=(Indirect&& other) { ptr = other.ptr.release(); return *this; }\n Indirect& operator=(const Indirect& other) { ptr = newOwned(*other.ptr); return *this; }\n\n bool operator==(const Indirect& other) const { return *ptr == *other.ptr; }\n bool operator!=(const Indirect& other) const { return *ptr != *other.ptr; }\n\n const T& operator*() const { return *ptr; }\n T& operator*() { return *ptr; }\n\n const T* operator->() const { return ptr.get(); }\n T* operator->() { return ptr.get(); }\n\nprivate:\n OwnedPtr ptr;\n};\n\n// TODO: Hide this somewhere private?\nclass Refcount {\npublic:\n Refcount(): strong(1), weak(0) {}\n Refcount(const Refcount& other) = delete;\n Refcount& operator=(const Refcount& other) = delete;\n\n static void inc(Refcount* r) {\n if (r != NULL) ++r->strong;\n }\n static bool dec(Refcount* r) {\n if (r == NULL) {\n return false;\n } else if (--r->strong == 0) {\n if (r->weak == 0) {\n delete r;\n }\n return true;\n } else {\n return false;\n }\n }\n\n static void incWeak(Refcount* r) {\n if (r != NULL) ++r->weak;\n }\n static void decWeak(Refcount* r) {\n if (r != NULL && --r->weak == 0 && r->strong == 0) {\n delete r;\n }\n }\n\n static bool release(Refcount* r) {\n if (r != NULL && r->strong == 1) {\n dec(r);\n return true;\n } else {\n return false;\n }\n }\n\n static bool isLive(Refcount* r) {\n return r != NULL && r->strong > 0;\n }\n\nprivate:\n int strong;\n int weak;\n};\n\ntemplate \nclass SmartPtr {\npublic:\n SmartPtr() : ptr(NULL), refcount(NULL) {}\n SmartPtr(std::nullptr_t) : ptr(NULL), refcount(NULL) {}\n ~SmartPtr() {\n if (Refcount::dec(refcount)) {\n deleteEnsuringCompleteType(ptr);\n }\n }\n\n SmartPtr(const SmartPtr& other) : ptr(other.ptr), refcount(other.refcount) {\n Refcount::inc(refcount);\n }\n template \n SmartPtr(const SmartPtr& other) : ptr(other.ptr), refcount(other.refcount) {\n Refcount::inc(refcount);\n }\n SmartPtr& operator=(const SmartPtr& other) {\n reset(other.ptr, other.refcount);\n return *this;\n }\n template \n SmartPtr& operator=(const SmartPtr& other) {\n reset(other.ptr, other.refcount);\n return *this;\n }\n\n SmartPtr(SmartPtr&& other) : ptr(other.ptr), refcount(other.refcount) {\n other.ptr = NULL;\n other.refcount = NULL;\n }\n template \n SmartPtr(SmartPtr&& other) : ptr(other.ptr), refcount(other.refcount) {\n other.ptr = NULL;\n other.refcount = NULL;\n }\n SmartPtr& operator=(SmartPtr&& other) {\n // Move pointers to locals before reset() in case &other == this.\n T* tempPtr = other.ptr;\n Refcount* tempRefcount = other.refcount;\n other.ptr = NULL;\n other.refcount = NULL;\n\n reset(NULL, NULL);\n\n ptr = tempPtr;\n refcount = tempRefcount;\n return *this;\n }\n template \n SmartPtr& operator=(SmartPtr&& other) {\n // Move pointers to locals before reset() in case &other == this.\n T* tempPtr = other.ptr;\n Refcount* tempRefcount = other.refcount;\n other.ptr = NULL;\n other.refcount = NULL;\n\n reset(NULL, NULL);\n\n ptr = tempPtr;\n refcount = tempRefcount;\n return *this;\n }\n\n template \n SmartPtr(OwnedPtr&& other)\n : ptr(other.releaseRaw()),\n refcount(ptr == NULL ? NULL : new Refcount()) {}\n template \n SmartPtr& operator=(OwnedPtr&& other) {\n reset(other.releaseRaw());\n return *this;\n }\n\n T* get() const { return ptr; }\n T* operator->() const { assert(ptr != NULL); return ptr; }\n T& operator*() const { assert(ptr != NULL); return *ptr; }\n\n template \n bool release(OwnedPtr* other) {\n if (Refcount::release(refcount)) {\n other->reset(ptr);\n ptr = NULL;\n return true;\n } else {\n return false;\n }\n }\n\n void clear() {\n reset(NULL);\n }\n\n bool operator==(const T* other) { return ptr == other; }\n bool operator!=(const T* other) { return ptr != other; }\n bool operator==(const SmartPtr& other) { return ptr == other.ptr; }\n bool operator!=(const SmartPtr& other) { return ptr != other.ptr; }\n\n void allocate() {\n reset(new T());\n }\n template \n void allocate(const P1& p1) {\n reset(new T(p1));\n }\n template \n void allocate(const P1& p1, const P2& p2) {\n reset(new T(p1, p2));\n }\n template \n void allocate(const P1& p1, const P2& p2, const P3& p3) {\n reset(new T(p1, p2, p3));\n }\n template \n void allocate(const P1& p1, const P2& p2, const P3& p3, const P4& p4) {\n reset(new T(p1, p2, p3, p4));\n }\n\n template \n void allocateSubclass() {\n reset(new Sub());\n }\n template \n void allocateSubclass(const P1& p1) {\n reset(new Sub(p1));\n }\n template \n void allocateSubclass(const P1& p1, const P2& p2) {\n reset(new Sub(p1, p2));\n }\n template \n void allocateSubclass(const P1& p1, const P2& p2, const P3& p3) {\n reset(new Sub(p1, p2, p3));\n }\n template \n void allocateSubclass(const P1& p1, const P2& p2, const P3& p3, const P4& p4) {\n reset(new Sub(p1, p2, p3, p4));\n }\n\nprivate:\n T* ptr;\n Refcount* refcount;\n\n inline void reset(T* newValue) {\n reset(newValue, newValue == NULL ? NULL : new Refcount());\n Refcount::dec(refcount);\n }\n\n void reset(T* newValue, Refcount* newRefcount) {\n T* oldValue = ptr;\n Refcount* oldRefcount = refcount;\n ptr = newValue;\n refcount = newRefcount;\n Refcount::inc(refcount);\n if (Refcount::dec(oldRefcount)) {\n deleteEnsuringCompleteType(oldValue);\n }\n }\n\n template \n friend class SmartPtr;\n template \n friend class WeakPtr;\n template \n friend class OwnedPtr;\n template \n friend class OwnedPtrVector;\n template \n friend class OwnedPtrQueue;\n template \n friend class OwnedPtrMap;\n};\n\ntemplate \nclass WeakPtr {\npublic:\n WeakPtr(): ptr(NULL), refcount(NULL) {}\n WeakPtr(const WeakPtr& other): ptr(other.ptr), refcount(other.refcount) {\n Refcount::incWeak(refcount);\n }\n WeakPtr(const SmartPtr& other): ptr(other.ptr), refcount(other.refcount) {\n Refcount::incWeak(refcount);\n }\n WeakPtr(std::nullptr_t): ptr(nullptr), refcount(nullptr) {}\n ~WeakPtr() {\n Refcount::decWeak(refcount);\n }\n\n WeakPtr& operator=(const WeakPtr& other) {\n Refcount* oldRefcount = refcount;\n ptr = other.ptr;\n refcount = other.refcount;\n Refcount::incWeak(refcount);\n Refcount::decWeak(oldRefcount);\n return *this;\n }\n template \n WeakPtr& operator=(const WeakPtr& other) {\n Refcount* oldRefcount = refcount;\n ptr = other.ptr;\n refcount = other.refcount;\n Refcount::incWeak(refcount);\n Refcount::decWeak(oldRefcount);\n return *this;\n }\n template \n WeakPtr& operator=(const SmartPtr& other) {\n Refcount* oldRefcount = refcount;\n ptr = other.ptr;\n refcount = other.refcount;\n Refcount::incWeak(refcount);\n Refcount::decWeak(oldRefcount);\n return *this;\n }\n WeakPtr& operator=(std::nullptr_t) {\n Refcount::decWeak(refcount);\n ptr = nullptr;\n refcount = nullptr;\n return *this;\n }\n\n template \n operator SmartPtr() const {\n SmartPtr result;\n if (Refcount::isLive(refcount)) {\n result.reset(ptr, refcount);\n }\n return result;\n }\n\nprivate:\n T* ptr;\n Refcount* refcount;\n};\n\ntemplate \nclass OwnedPtrVector {\npublic:\n OwnedPtrVector() {}\n OwnedPtrVector(const OwnedPtrVector&) = delete;\n OwnedPtrVector(OwnedPtrVector&& other) {\n vec.swap(other.vec);\n }\n ~OwnedPtrVector() {\n for (typename std::vector::const_iterator iter = vec.begin(); iter != vec.end(); ++iter) {\n deleteEnsuringCompleteType(*iter);\n }\n }\n\n OwnedPtrVector& operator=(const OwnedPtrVector&) = delete;\n\n int size() const { return vec.size(); }\n T* get(int index) const { return vec[index]; }\n bool empty() const { return vec.empty(); }\n\n void add(OwnedPtr ptr) {\n vec.push_back(ptr.releaseRaw());\n }\n\n void set(int index, OwnedPtr ptr) {\n deleteEnsuringCompleteType(vec[index]);\n vec[index] = ptr->releaseRaw();\n }\n\n OwnedPtr release(int index) {\n T* result = vec[index];\n vec[index] = NULL;\n return OwnedPtr(result);\n }\n\n OwnedPtr releaseBack() {\n T* result = vec.back();\n vec.pop_back();\n return OwnedPtr(result);\n }\n\n OwnedPtr releaseAndShift(int index) {\n T* result = vec[index];\n vec.erase(vec.begin() + index);\n return OwnedPtr(result);\n }\n\n void clear() {\n for (typename std::vector::const_iterator iter = vec.begin(); iter != vec.end(); ++iter) {\n deleteEnsuringCompleteType(*iter);\n }\n vec.clear();\n }\n\n void swap(OwnedPtrVector* other) {\n vec.swap(other->vec);\n }\n\n class Appender {\n public:\n explicit Appender(OwnedPtrVector* vec) : vec(vec) {}\n\n void add(OwnedPtr ptr) {\n vec->add(ptr.release());\n }\n\n private:\n OwnedPtrVector* vec;\n };\n\n Appender appender() {\n return Appender(this);\n }\n\nprivate:\n std::vector vec;\n};\n\ntemplate \nclass OwnedPtrDeque {\npublic:\n OwnedPtrDeque() {}\n ~OwnedPtrDeque() {\n for (typename std::deque::const_iterator iter = q.begin(); iter != q.end(); ++iter) {\n deleteEnsuringCompleteType(*iter);\n }\n }\n\n int size() const { return q.size(); }\n T* get(int index) const { return q[index]; }\n bool empty() const { return q.empty(); }\n\n void pushFront(OwnedPtr ptr) {\n q.push_front(ptr.releaseRaw());\n }\n\n OwnedPtr popFront() {\n T* ptr = q.front();\n q.pop_front();\n return OwnedPtr(ptr);\n }\n\n void pushBack(OwnedPtr ptr) {\n q.push_back(ptr.releaseRaw());\n }\n\n OwnedPtr popBack() {\n T* ptr = q.back();\n q.pop_back();\n return OwnedPtr(ptr);\n }\n\n OwnedPtr releaseAndShift(int index) {\n T* ptr = q[index];\n q.erase(q.begin() + index);\n return OwnedPtr(ptr);\n }\n\n void clear() {\n for (typename std::deque::const_iterator iter = q.begin(); iter != q.end(); ++iter) {\n deleteEnsuringCompleteType(*iter);\n }\n q.clear();\n }\n\n void swap(OwnedPtrDeque* other) {\n q.swap(other->q);\n }\n\nprivate:\n std::deque q;\n};\n\ntemplate \nclass OwnedPtrQueue {\npublic:\n OwnedPtrQueue() {}\n ~OwnedPtrQueue() {\n clear();\n }\n\n int size() const { return q.size(); }\n bool empty() const { return q.empty(); }\n\n void push(OwnedPtr ptr) {\n q.push(ptr.releaseRaw());\n }\n\n OwnedPtr pop() {\n T* ptr = q.front();\n q.pop();\n return OwnedPtr(ptr);\n }\n\n void clear() {\n while (!q.empty()) {\n deleteEnsuringCompleteType(q.front());\n q.pop();\n }\n }\n\n class Appender {\n public:\n Appender(OwnedPtrQueue* q) : q(q) {}\n\n void add(OwnedPtr ptr) {\n q->push(ptr);\n }\n\n private:\n OwnedPtrQueue* q;\n };\n\n Appender appender() {\n return Appender(this);\n }\n\nprivate:\n std::queue q;\n};\n\ntemplate ,\n typename EqualsFunc = std::equal_to >\nclass OwnedPtrMap {\n typedef std::unordered_map InnerMap;\n\npublic:\n OwnedPtrMap() {}\n ~OwnedPtrMap() {\n for (typename InnerMap::const_iterator iter = map.begin();\n iter != map.end(); ++iter) {\n deleteEnsuringCompleteType(iter->second);\n }\n }\n\n bool empty() const {\n return map.empty();\n }\n\n int size() const {\n return map.size();\n }\n\n bool contains(const Key& key) const {\n return map.count(key) > 0;\n }\n\n T* get(const Key& key) const {\n typename InnerMap::const_iterator iter = map.find(key);\n if (iter == map.end()) {\n return NULL;\n } else {\n return iter->second;\n }\n }\n\n void add(const Key& key, OwnedPtr ptr) {\n T* value = ptr.releaseRaw();\n std::pair insertResult =\n map.insert(std::make_pair(key, value));\n if (!insertResult.second) {\n deleteEnsuringCompleteType(insertResult.first->second);\n insertResult.first->second = value;\n }\n }\n\n bool addIfNew(const Key& key, OwnedPtr ptr) {\n T* value = ptr.releaseRaw();\n std::pair insertResult =\n map.insert(std::make_pair(key, value));\n if (insertResult.second) {\n return true;\n } else {\n deleteEnsuringCompleteType(value);\n return false;\n }\n }\n\n bool release(const Key& key, OwnedPtr* output) {\n typename InnerMap::iterator iter = map.find(key);\n if (iter == map.end()) {\n output->reset(NULL);\n return false;\n } else {\n output->reset(iter->second);\n map.erase(iter);\n return true;\n }\n }\n\n void releaseAll(typename OwnedPtrVector::Appender output) {\n for (typename InnerMap::const_iterator iter = map.begin();\n iter != map.end(); ++iter) {\n output.add(OwnedPtr(iter->second));\n }\n map.clear();\n }\n\n bool erase(const Key& key) {\n typename InnerMap::iterator iter = map.find(key);\n if (iter == map.end()) {\n return false;\n } else {\n deleteEnsuringCompleteType(iter->second);\n map.erase(iter);\n return true;\n }\n }\n\n void clear() {\n for (typename InnerMap::const_iterator iter = map.begin();\n iter != map.end(); ++iter) {\n deleteEnsuringCompleteType(iter->second);\n }\n map.clear();\n }\n\n void swap(OwnedPtrMap* other) {\n map.swap(other->map);\n }\n\n class Iterator {\n public:\n Iterator(const OwnedPtrMap& map)\n : nextIter(map.map.begin()),\n end(map.map.end()) {}\n\n bool next() {\n if (nextIter == end) {\n return false;\n } else {\n iter = nextIter;\n ++nextIter;\n return true;\n }\n }\n\n const Key& key() {\n return iter->first;\n }\n\n T* value() {\n return iter->second;\n }\n\n private:\n typename InnerMap::const_iterator iter;\n typename InnerMap::const_iterator nextIter;\n typename InnerMap::const_iterator end;\n };\n\nprivate:\n InnerMap map;\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_BASE_OWNEDPTR_H_\n" }, { "path": "src/base/Promise.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"Promise.h\"\n\nnamespace ekam {\n\nRunnable::~Runnable() {}\nPendingRunnable::~PendingRunnable() {}\nExecutor::~Executor() noexcept(false) {}\n\n} // namespace ekam\n" }, { "path": "src/base/Promise.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_BASE_PROMISE_H_\n#define KENTONSCODE_BASE_PROMISE_H_\n\n#include \n#include \n#include \n#include \n#include \n#include \n#include \n\n#include \"OwnedPtr.h\"\n#include \"Debug.h\"\n\nnamespace ekam {\n\n// =======================================================================================\n// TODO: Put elsewhere\n\nenum class Void {\n VOID\n};\n\ntemplate \nclass ValuePack;\n\ntemplate \nclass ValuePack: private ValuePack {\npublic:\n ValuePack(Head&& head, Tail&&... tail)\n : ValuePack(std::forward(tail)...),\n head(std::forward(head)) {}\n ValuePack(const ValuePack& other) = delete;\n ValuePack(ValuePack&& other) = default;\n\n ValuePack& operator=(const ValuePack& other) = delete;\n ValuePack& operator=(ValuePack&& other) {\n head = std::move(other.head);\n ValuePack::operator=(std::move(other));\n }\n\n template \n auto apply(const Func& func, Params&&... params) const ->\n decltype(func(std::declval()..., std::declval(),\n std::declval()...)) {\n return ValuePack::apply(func, std::forward(params)..., head);\n }\n\n template \n auto applyMoving(const Func& func, Params&&... params) ->\n decltype(func(std::declval()..., std::declval(), std::declval()...)) {\n return ValuePack::applyMoving(func, std::forward(params)..., std::move(head));\n }\n\nprivate:\n Head head;\n};\n\ntemplate <>\nclass ValuePack<> {\npublic:\n ValuePack() {}\n\n template \n auto apply(const Func& func, Params&&... params) const ->\n decltype(func(std::declval()...)) {\n func(std::forward(params)...);\n }\n\n template \n auto applyMoving(const Func& func, Params&&... params) ->\n decltype(func(std::declval()...)) {\n func(std::forward(params)...);\n }\n};\n\n// =======================================================================================\n// TODO: Put elsewhere\n\nclass WeakLink {\npublic:\n WeakLink(): other(nullptr) {}\n WeakLink(WeakLink* other): other(other) {\n if (other != nullptr) {\n other->disentangle();\n other->other = this;\n }\n }\n ~WeakLink() {\n disentangle();\n }\n\n void entangle(WeakLink* other) {\n disentangle();\n this->other = other;\n if (other != nullptr) {\n other->disentangle();\n other->other = this;\n }\n }\n\n bool isEntangled() {\n return other != nullptr;\n }\n\nprivate:\n WeakLink* other;\n\n void disentangle() {\n if (other != nullptr) {\n other->other = nullptr;\n other = nullptr;\n }\n }\n};\n\n// =======================================================================================\n\ntemplate \nclass Promise;\n\nclass Runnable {\npublic:\n virtual ~Runnable();\n\n virtual void run() = 0;\n};\n\nclass PendingRunnable {\npublic:\n virtual ~PendingRunnable();\n};\n\nclass Executor {\npublic:\n virtual ~Executor() noexcept(false);\n\n virtual OwnedPtr runLater(OwnedPtr runnable) = 0;\n\n template \n class When;\n\n template \n When::type...> when(Types&&... params);\n};\n\ntemplate \nclass LambdaRunnable: public Runnable {\npublic:\n LambdaRunnable(Func&& func): func(std::move(func)) {}\n ~LambdaRunnable() {}\n\n void run() {\n func();\n }\n\nprivate:\n Func func;\n};\n\ntemplate \nOwnedPtr newLambdaRunnable(Func&& func) {\n return newOwned>(std::move(func));\n}\n\n// =======================================================================================\n\ntemplate \nclass MaybeException {\npublic:\n MaybeException(): broken(false) {\n new(&this->value) T;\n }\n MaybeException(T value): broken(false) {\n new(&this->value) T(std::move(value));\n }\n MaybeException(std::exception_ptr error): broken(true) {\n new(&this->error) std::exception_ptr(std::move(error));\n }\n MaybeException(const MaybeException& other) {\n if (broken) {\n new(&error) std::exception_ptr(other.error);\n } else {\n new(&value) T(other.value);\n }\n }\n MaybeException(MaybeException&& other): broken(other.broken) {\n if (broken) {\n new(&error) std::exception_ptr(std::move(other.error));\n } else {\n new(&value) T(std::move(other.value));\n }\n }\n\n ~MaybeException() {\n if (broken) {\n error.~exception_ptr();\n } else {\n value.~T();\n }\n }\n\n MaybeException& operator=(const MaybeException& other) {\n this->~MaybeException();\n broken = other.broken;\n if (broken) {\n new(&error) std::exception_ptr(other.error);\n } else {\n new(&value) T(other.value);\n }\n return *this;\n }\n MaybeException& operator=(MaybeException&& other) {\n this->~MaybeException();\n broken = other.broken;\n if (broken) {\n new(&error) std::exception_ptr(std::move(other.error));\n } else {\n new(&value) T(std::move(other.value));\n }\n return *this;\n }\n\n bool isException() {\n return broken;\n }\n\n const T& get() {\n if (broken) {\n std::rethrow_exception(error);\n }\n return value;\n }\n\n T release() {\n if (broken) {\n std::rethrow_exception(error);\n }\n return std::move(value);\n }\n\nprivate:\n bool broken;\n union {\n T value;\n std::exception_ptr error;\n };\n};\n\ntemplate <>\nclass MaybeException {\npublic:\n MaybeException(): broken(false) {}\n MaybeException(std::exception_ptr error): broken(true) {\n new(&this->error) std::exception_ptr(std::move(error));\n }\n MaybeException(const MaybeException& other) {\n if (broken) {\n new(&error) std::exception_ptr(other.error);\n }\n }\n MaybeException(MaybeException&& other): broken(other.broken) {\n if (broken) {\n new(&error) std::exception_ptr(std::move(other.error));\n }\n }\n\n ~MaybeException() {\n if (broken) {\n error.~exception_ptr();\n }\n }\n\n MaybeException& operator=(const MaybeException& other) {\n this->~MaybeException();\n broken = other.broken;\n if (broken) {\n new(&error) std::exception_ptr(other.error);\n }\n return *this;\n }\n MaybeException& operator=(MaybeException&& other) {\n this->~MaybeException();\n broken = other.broken;\n if (broken) {\n new(&error) std::exception_ptr(std::move(other.error));\n }\n return *this;\n }\n\n bool isException() {\n return broken;\n }\n\n Void get() {\n if (broken) {\n std::rethrow_exception(error);\n }\n return Void::VOID;\n }\n\n Void release() {\n if (broken) {\n std::rethrow_exception(error);\n }\n return Void::VOID;\n }\n\nprivate:\n bool broken;\n union {\n std::exception_ptr error;\n };\n};\n\n// =======================================================================================\n\ntemplate \nclass PromiseFulfiller {\npublic:\n typedef T PromiseType;\n\n class Callback;\n};\n\nnamespace promiseInternal {\n\nstruct PromiseConstructors;\n\ntemplate \nclass DependentPromiseFulfiller;\n\nclass PromiseListener {\npublic:\n virtual void dependencyDone(bool failed) = 0;\n};\n\ntemplate \nclass PromiseState {\npublic:\n PromiseState(): owner(nullptr), listener(nullptr), fulfilled(false), failed(false) {}\n virtual ~PromiseState() {}\n\n Promise* owner;\n OwnedPtr chainedPromise;\n\n // (effectively) private:\n PromiseListener* listener;\n bool fulfilled;\n bool failed;\n MaybeException value;\n\n void setListener(PromiseListener* listener) {\n if (this->listener != nullptr) {\n throw std::invalid_argument(\"Already waiting on this Promise.\");\n }\n this->listener = listener;\n if (fulfilled) {\n listener->dependencyDone(failed);\n }\n }\n\n void preFulfill() {\n if (fulfilled) {\n throw std::logic_error(\"Already fulfilled this promise.\");\n }\n fulfilled = true;\n }\n\n void postFulfill(bool failed) {\n this->failed = failed;\n if (listener != nullptr) {\n listener->dependencyDone(failed);\n }\n }\n\n void fulfill(const T& value) {\n preFulfill();\n this->value = value;\n postFulfill(false);\n }\n\n void fulfill(T&& value) {\n preFulfill();\n this->value = std::move(value);\n postFulfill(false);\n }\n\n void fulfill(Promise chainedPromise);\n\n void propagateCurrentException() {\n preFulfill();\n this->value = std::current_exception();\n postFulfill(true);\n }\n\n T release() {\n return value.release();\n }\n\n MaybeException releaseMaybeException() {\n return std::move(value);\n }\n\n template \n friend class DependentPromiseFulfiller;\n friend struct PromiseConstructors;\n};\n\ntemplate <>\nclass PromiseState {\npublic:\n PromiseState(): owner(nullptr), listener(nullptr), fulfilled(false), failed(false) {}\n virtual ~PromiseState() {}\n\n Promise* owner;\n OwnedPtr chainedPromise;\n\nprivate:\n PromiseListener* listener;\n bool fulfilled;\n bool failed;\n MaybeException value;\n\n void setListener(PromiseListener* listener) {\n if (this->listener != nullptr) {\n throw std::invalid_argument(\"Already waiting on this Promise.\");\n }\n this->listener = listener;\n if (fulfilled) {\n listener->dependencyDone(failed);\n }\n }\n\n void preFulfill() {\n if (fulfilled) {\n throw std::logic_error(\"Already fulfilled this promise.\");\n }\n fulfilled = true;\n }\n\n void postFulfill(bool failed) {\n this->failed = failed;\n if (listener != nullptr) {\n listener->dependencyDone(failed);\n }\n }\n\n void fulfill() {\n preFulfill();\n postFulfill(false);\n }\n\n void fulfill(Promise chainedPromise);\n\n void propagateCurrentException() {\n preFulfill();\n this->value = std::current_exception();\n postFulfill(true);\n }\n\n Void release() {\n return value.release();\n }\n\n MaybeException releaseMaybeException() {\n return std::move(value);\n }\n\n friend class PromiseFulfiller::Callback;\n template \n friend class DependentPromiseFulfiller;\n friend struct PromiseConstructors;\n};\n\ntemplate \nstruct Unpack {\n typedef T Type;\n};\n\ntemplate \nstruct Unpack> {\n typedef T Type;\n};\n\ntemplate <>\nstruct Unpack> {\n typedef Void Type;\n};\n\ntemplate \nstruct Chain {\n typedef T Type;\n};\n\ntemplate \nstruct Chain> {\n typedef T Type;\n};\n\n// =======================================================================================\n\ntemplate \nclass FulfillerPromiseState;\n\n} // namespace promiseInternal\n\ntemplate \nclass PromiseFulfiller::Callback {\npublic:\n template \n void fulfill(U&& value) {\n state->fulfill(std::forward(value));\n }\n\n void propagateCurrentException() {\n state->propagateCurrentException();\n }\n\nprivate:\n promiseInternal::PromiseState* state;\n\n Callback(promiseInternal::PromiseState* state): state(state) {}\n ~Callback() {}\n\n template \n friend class promiseInternal::FulfillerPromiseState;\n};\n\ntemplate <>\nclass PromiseFulfiller::Callback {\npublic:\n void fulfill() {\n state->fulfill();\n }\n\n void fulfill(Promise chain);\n\n void propagateCurrentException() {\n state->propagateCurrentException();\n }\n\nprivate:\n promiseInternal::PromiseState* state;\n\n Callback(promiseInternal::PromiseState* state): state(state) {}\n ~Callback() {}\n\n template \n friend class promiseInternal::FulfillerPromiseState;\n};\n\nnamespace promiseInternal {\n\ntemplate \nclass FulfillerPromiseState : public PromiseState {\npublic:\n template \n FulfillerPromiseState(Params&&... params)\n : PromiseState(),\n callback(this),\n fulfillerImpl(&callback, std::forward(params)...) {}\n virtual ~FulfillerPromiseState() {}\n\nprivate:\n typename PromiseFulfiller::Callback callback;\n PromiseFulfillerImpl fulfillerImpl;\n};\n\n// =======================================================================================\n\ntemplate \nstruct CallAndFulfillFunctor {\n template \n void operator()(WeakLink* linkToFulfiller, Callback* callback,\n Func2& func, Params&&... params) const {\n try {\n auto result = func(std::forward(params)...);\n if (linkToFulfiller->isEntangled()) {\n callback->fulfill(std::move(result));\n }\n } catch (...) {\n callback->propagateCurrentException();\n }\n }\n};\n\ntemplate <>\nstruct CallAndFulfillFunctor {\n template \n void operator()(WeakLink* linkToFulfiller, Callback* callback,\n Func2& func, Params&&... params) const {\n try {\n func(std::forward(params)...);\n if (linkToFulfiller->isEntangled()) {\n callback->fulfill();\n }\n } catch (...) {\n callback->propagateCurrentException();\n }\n }\n};\n\ntemplate \nclass DependentPromiseFulfiller : public PromiseFulfiller, public PromiseListener {\npublic:\n typedef typename PromiseFulfiller::Callback Callback;\n\n DependentPromiseFulfiller(Callback* callback, Executor* executor, Func&& func,\n ExceptionHandler&& exceptionHandler, ParamPack&& params)\n : callback(callback),\n executor(executor),\n func(std::move(func)),\n exceptionHandler(std::move(exceptionHandler)),\n params(std::move(params)),\n dependencyFailed(false),\n dependenciesLeft(1) {\n addAllDependencies();\n }\n\n ~DependentPromiseFulfiller() {\n if (pendingReadyLater != nullptr) {\n DEBUG_INFO << \"Promise canceled: \" << this;\n }\n }\n\n void dependencyDone(bool failed) {\n if (failed) {\n dependencyFailed = true;\n }\n if (--dependenciesLeft == 0) {\n readyLater();\n }\n }\n\n Callback* callback;\n Executor* executor;\n Func func;\n ExceptionHandler exceptionHandler;\n ParamPack params;\n\nprivate:\n bool dependencyFailed;\n int dependenciesLeft;\n OwnedPtr pendingReadyLater;\n WeakLink weakLink;\n\n struct AddDependenciesFunctor {\n template \n void operator()(DependentPromiseFulfiller* fulfiller, const Promise& first,\n Rest&&... rest) const {\n ++fulfiller->dependenciesLeft;\n first.state->setListener(fulfiller);\n operator()(fulfiller, std::forward(rest)...);\n }\n\n template \n void operator()(DependentPromiseFulfiller* fulfiller, First&& first, Rest&&... rest) const {\n operator()(fulfiller, std::forward(rest)...);\n }\n\n void operator()(DependentPromiseFulfiller* fulfiller) const {\n if (--fulfiller->dependenciesLeft == 0) {\n fulfiller->readyLater();\n }\n }\n };\n\n void addAllDependencies() {\n params.apply(AddDependenciesFunctor(), this);\n }\n\n template \n static U&& unpack(U&& value) {\n return std::forward(value);\n }\n\n template \n static typename Unpack>::Type unpack(Promise&& promise) {\n return promise.state->release();\n }\n\n struct DoReadyFunctor {\n template \n void operator()(WeakLink* linkToFulfiller, Callback* callback,\n Func& func, Params&&... params) const {\n CallAndFulfillFunctor(params))...))>()(\n linkToFulfiller, callback, func, unpack(std::forward(params))...);\n }\n };\n\n void ready() {\n // Move stuff to stack in case promise is deleted during callback.\n ParamPack localParams(std::move(params));\n Func localFunc(std::move(func));\n\n WeakLink linkToFulfiller(&weakLink);\n localParams.applyMoving(DoReadyFunctor(), &linkToFulfiller, callback, localFunc);\n }\n\n template \n static U&& unpackMaybeException(U&& value) {\n return std::forward(value);\n }\n\n template \n static MaybeException unpackMaybeException(Promise&& promise) {\n return promise.state->releaseMaybeException();\n }\n\n struct DoErrorFunctor {\n template \n void operator()(WeakLink* linkToFulfiller, Callback* callback,\n const ExceptionHandler& exceptionHandler, Params&&... params) const {\n CallAndFulfillFunctor(params))...))>()(\n linkToFulfiller, callback, exceptionHandler,\n unpackMaybeException(std::forward(params))...);\n }\n };\n\n void error() {\n // Move stuff to stack in case promise is deleted during callback.\n ParamPack localParams(std::move(this->params));\n ExceptionHandler localExceptionHandler(std::move(this->exceptionHandler));\n\n WeakLink linkToFulfiller(&weakLink);\n localParams.applyMoving(DoErrorFunctor(), &linkToFulfiller, callback, localExceptionHandler);\n }\n\n void readyLater() {\n pendingReadyLater = executor->runLater(newLambdaRunnable(\n [this]() {\n auto objectToDeleteOnReturn = this->pendingReadyLater.release();\n if (this->dependencyFailed) {\n this->error();\n } else {\n this->ready();\n }\n }));\n }\n};\n\n// =======================================================================================\n\ntemplate \nstruct ForceErrorFunctor {\n template \n ReturnType operator()(Head&& head, Tail&&... tail) const {\n return operator()(tail...);\n }\n template \n ReturnType operator()(MaybeException&& head, Tail&&... tail) const {\n head.get();\n return operator()(tail...);\n }\n ReturnType operator()() const {\n throw std::logic_error(\n \"Promise implementation failure: Exception handler called with no exceptions.\");\n }\n};\n\n} // namespace promiseInternal\n\n// =======================================================================================\n\ntemplate \nclass Promise {\npublic:\n Promise() {}\n Promise(const Promise& other) = delete;\n Promise(Promise&& other): state(other.state.release()) {\n if (state != nullptr) {\n state->owner = this;\n }\n }\n\n Promise& operator=(const Promise& other) = delete;\n Promise& operator=(Promise&& other) {\n state = other.state.release();\n state->owner = this;\n return *this;\n }\n\n bool operator==(std::nullptr_t) {\n return state == nullptr;\n }\n bool operator!=(std::nullptr_t) {\n return state != nullptr;\n }\n\n T* operator->() const {\n return state->getValue()->operator->();\n }\n\n Promise release() {\n return std::move(*this);\n }\n\nprivate:\n typedef promiseInternal::PromiseState State;\n\n explicit Promise(OwnedPtr state): state(state.release()) {\n while (this->state->chainedPromise != nullptr) {\n this->state = this->state->chainedPromise.release();\n }\n this->state->owner = this;\n }\n\n OwnedPtr state;\n\n friend class Executor;\n friend struct promiseInternal::PromiseConstructors;\n template \n friend class promiseInternal::DependentPromiseFulfiller;\n friend class promiseInternal::PromiseState;\n};\n\ninline void PromiseFulfiller::Callback::fulfill(Promise chain) {\n state->fulfill(chain.release());\n}\n\nnamespace promiseInternal {\n\ntemplate \nvoid PromiseState::fulfill(Promise chainedPromise) {\n auto listener = this->listener;\n auto owner = this->owner;\n\n if (owner == nullptr) {\n throw std::logic_error(\n \"Not supported: Calling fulfill() with a chained promise from the PromiseFulfiller's \"\n \"constructor. This is probably not what you intended anyway.\");\n } else {\n *owner = chainedPromise.release(); // will delete this!\n if (listener != nullptr) {\n owner->state->setListener(listener);\n }\n }\n}\n\ninline void PromiseState::fulfill(Promise chainedPromise) {\n auto listener = this->listener;\n auto owner = this->owner;\n\n if (owner == nullptr) {\n throw std::logic_error(\n \"Not supported: Calling fulfill() with a chained promise from the PromiseFulfiller's \"\n \"constructor. This is probably not what you intended anyway.\");\n } else {\n *owner = chainedPromise.release(); // will delete this!\n if (listener != nullptr) {\n owner->state->setListener(listener);\n }\n }\n}\n\nstruct PromiseConstructors {\n template \n static Promise newPromise(Params&&... params) {\n typedef typename PromiseFulfillerImpl::PromiseType T;\n return Promise(newOwned>(\n std::forward(params)...));\n }\n\n template \n static Promise::type> newFulfilledPromise(T&& value) {\n Promise result(newOwned>());\n result.state->fulfill(std::forward(value));\n return result;\n }\n\n static Promise newFulfilledPromise() {\n Promise result(newOwned>());\n result.state->fulfill();\n return result;\n }\n\n template \n static Promise newPromiseFromCurrentException() {\n Promise result(newOwned>());\n result.state->propagateCurrentException();\n return result;\n }\n};\n\n} // namespace promiseInternal\n\ntemplate \nPromise newPromise(Params&&... params) {\n return promiseInternal::PromiseConstructors::newPromise(\n std::forward(params)...);\n}\n\ntemplate \nPromise::type> newFulfilledPromise(T&& value) {\n return promiseInternal::PromiseConstructors::newFulfilledPromise(std::forward(value));\n}\n\ninline Promise newFulfilledPromise() {\n return promiseInternal::PromiseConstructors::newFulfilledPromise();\n}\n\ntemplate \nPromise newPromiseFromCurrentException() {\n return promiseInternal::PromiseConstructors::newPromiseFromCurrentException();\n}\n\n// =======================================================================================\n\ntemplate \nclass Executor::When {\n typedef ValuePack ParamPack;\npublic:\n When(const When& other) = delete;\n When(When&& other) = default;\n\n When& operator=(const When& other) = delete;\n When& operator=(When&& other) = delete;\n\n#define RESULT_TYPE \\\n typename promiseInternal::Chain< \\\n decltype(func(std::declval::Type>()...))>::Type\n\n template \n auto operator()(Func&& func) -> Promise {\n typedef RESULT_TYPE ResultType;\n return operator()(std::move(func), promiseInternal::ForceErrorFunctor());\n }\n\n template \n auto operator()(Func&& func, ExceptionHandler&& exceptionHandler) -> Promise {\n typedef RESULT_TYPE ResultType;\n\n typedef promiseInternal::DependentPromiseFulfiller<\n ResultType, Func, ExceptionHandler, ParamPack> Fulfiller;\n\n return newPromise(\n executor, std::move(func), std::move(exceptionHandler), std::move(params));\n }\n\n#undef RESULT_TYPE\n\nprivate:\n Executor* executor;\n ParamPack params;\n\n When(Executor* executor, Types&&... params)\n : executor(executor),\n params(std::move(params)...) {}\n friend class Executor;\n};\n\ntemplate \nExecutor::When::type...> Executor::when(Types&&... params) {\n return When::type...>(this, std::move(params)...);\n}\n\n} // namespace ekam\n\n#endif // KENTONSCODE_BASE_PROMISE_H_\n" }, { "path": "src/base/Promise_test.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"Promise.h\"\n#include \n#include \n#include \n\nnamespace ekam {\nnamespace {\n\nclass MockExecutor: public Executor {\nprivate:\n class PendingRunnableImpl : public PendingRunnable {\n public:\n PendingRunnableImpl(MockExecutor* executor, OwnedPtr runnable)\n : executor(executor), runnable(runnable.release()) {}\n\n ~PendingRunnableImpl() {\n if (runnable != nullptr) {\n for (auto iter = executor->queue.begin(); iter != executor->queue.end(); ++iter) {\n if (*iter == this) {\n executor->queue.erase(iter);\n break;\n }\n }\n }\n }\n\n void run() {\n runnable.release()->run();\n }\n\n private:\n MockExecutor* executor;\n OwnedPtr runnable;\n };\n\npublic:\n MockExecutor() {}\n ~MockExecutor() {\n EXPECT_TRUE(queue.empty());\n }\n\n void runNext() {\n ASSERT_FALSE(queue.empty());\n auto front = queue.front();\n queue.pop_front();\n front->run();\n }\n\n bool empty() {\n return queue.empty();\n }\n\n // implements Executor -----------------------------------------------------------------\n OwnedPtr runLater(OwnedPtr runnable) {\n auto result = newOwned(this, runnable.release());\n queue.push_back(result.get());\n return result.release();\n }\n\nprivate:\n std::deque queue;\n};\n\ntemplate \nclass MockPromiseFulfiller: public PromiseFulfiller {\npublic:\n typedef typename PromiseFulfiller::Callback Callback;\n MockPromiseFulfiller(Callback* callback, Callback** callbackPtr)\n : callbackPtr(callbackPtr) {\n *callbackPtr = callback;\n }\n ~MockPromiseFulfiller() {\n *callbackPtr = nullptr;\n }\n\nprivate:\n Callback** callbackPtr;\n};\n\nTEST(PromiseTest, Basic) {\n MockExecutor mockExecutor;\n\n PromiseFulfiller::Callback* fulfiller;\n auto promise = newPromise>(&fulfiller);\n\n bool triggered = false;\n\n Promise promise2 = mockExecutor.when(promise)(\n [&triggered](int i) -> int {\n EXPECT_EQ(5, i);\n triggered = true;\n return 123;\n });\n\n EXPECT_FALSE(triggered);\n\n fulfiller->fulfill(5);\n\n EXPECT_FALSE(triggered);\n\n mockExecutor.runNext();\n\n EXPECT_TRUE(triggered);\n\n // Fulfiller deleted because promise has been consumed.\n EXPECT_TRUE(fulfiller == nullptr);\n}\n\nTEST(PromiseTest, PreFulfilled) {\n MockExecutor mockExecutor;\n\n auto promise = newFulfilledPromise(5);\n\n bool triggered = false;\n\n Promise promise2 = mockExecutor.when(promise)(\n [&triggered](int i) -> int {\n EXPECT_EQ(5, i);\n triggered = true;\n return 123;\n });\n\n EXPECT_FALSE(triggered);\n\n mockExecutor.runNext();\n\n EXPECT_TRUE(triggered);\n}\n\nTEST(PromiseTest, Dependent) {\n MockExecutor mockExecutor;\n\n PromiseFulfiller::Callback* fulfiller1;\n auto promise1 = newPromise>(&fulfiller1);\n PromiseFulfiller::Callback* fulfiller2;\n auto promise2 = newPromise>(&fulfiller2);\n\n Promise promise3 = mockExecutor.when(promise1, promise2)(\n [](int a, int b) -> int {\n return a + b;\n });\n\n int result = 0;\n\n Promise promise4 = mockExecutor.when(promise3)(\n [&result](int a) {\n result = a;\n });\n\n EXPECT_TRUE(mockExecutor.empty());\n fulfiller1->fulfill(12);\n EXPECT_TRUE(mockExecutor.empty());\n fulfiller2->fulfill(34);\n EXPECT_FALSE(mockExecutor.empty());\n mockExecutor.runNext();\n EXPECT_FALSE(mockExecutor.empty());\n mockExecutor.runNext();\n EXPECT_EQ(result, 46);\n}\n\nTEST(PromiseTest, Chained) {\n MockExecutor mockExecutor;\n\n PromiseFulfiller::Callback* fulfiller1;\n auto promise1 = newPromise>(&fulfiller1);\n PromiseFulfiller::Callback* fulfiller2;\n auto promise2 = newPromise>(&fulfiller2);\n\n int result = 0;\n\n Promise promise3 = mockExecutor.when(promise2)(\n [&result](int a) {\n result = a;\n });\n\n EXPECT_TRUE(mockExecutor.empty());\n fulfiller2->fulfill(promise1.release());\n EXPECT_TRUE(mockExecutor.empty());\n EXPECT_EQ(0, result);\n fulfiller1->fulfill(123);\n EXPECT_FALSE(mockExecutor.empty());\n mockExecutor.runNext();\n EXPECT_EQ(result, 123);\n}\n\nTEST(PromiseTest, ChainedVoid) {\n MockExecutor mockExecutor;\n\n PromiseFulfiller::Callback* fulfiller1;\n auto promise1 = newPromise>(&fulfiller1);\n PromiseFulfiller::Callback* fulfiller2;\n auto promise2 = newPromise>(&fulfiller2);\n\n bool triggered = false;\n\n Promise promise3 = mockExecutor.when(promise2)(\n [&triggered](Void) {\n triggered = true;\n });\n\n EXPECT_TRUE(mockExecutor.empty());\n fulfiller2->fulfill(promise1.release());\n EXPECT_TRUE(mockExecutor.empty());\n EXPECT_FALSE(triggered);\n fulfiller1->fulfill();\n EXPECT_FALSE(mockExecutor.empty());\n mockExecutor.runNext();\n EXPECT_TRUE(triggered);\n}\n\nTEST(PromiseTest, ChainedVoidWhen) {\n MockExecutor mockExecutor;\n\n PromiseFulfiller::Callback* fulfiller1;\n auto promise1 = newPromise>(&fulfiller1);\n PromiseFulfiller::Callback* fulfiller2 = nullptr;\n\n Promise promise3 = mockExecutor.when(promise1)(\n [&fulfiller2](Void) -> Promise {\n DEBUG_ERROR << \"promise3\";\n return newPromise>(&fulfiller2);\n });\n\n bool triggered = false;\n\n Promise promise4 = mockExecutor.when(promise3)(\n [&triggered](Void) {\n DEBUG_ERROR << \"promise4\";\n triggered = true;\n });\n\n EXPECT_TRUE(mockExecutor.empty());\n fulfiller1->fulfill();\n EXPECT_FALSE(mockExecutor.empty());\n EXPECT_TRUE(fulfiller2 == nullptr);\n mockExecutor.runNext();\n EXPECT_FALSE(fulfiller2 == nullptr);\n EXPECT_TRUE(mockExecutor.empty());\n EXPECT_FALSE(triggered);\n\n ASSERT_TRUE(fulfiller2 != nullptr);\n fulfiller2->fulfill();\n EXPECT_FALSE(mockExecutor.empty());\n mockExecutor.runNext();\n EXPECT_TRUE(triggered);\n}\n\nTEST(PromiseTest, ChainedPreFulfilled) {\n MockExecutor mockExecutor;\n\n auto promise1 = newFulfilledPromise(123);\n PromiseFulfiller::Callback* fulfiller2;\n auto promise2 = newPromise>(&fulfiller2);\n\n int result = 0;\n\n Promise promise3 = mockExecutor.when(promise2)(\n [&result](int a) {\n result = a;\n });\n\n EXPECT_TRUE(mockExecutor.empty());\n fulfiller2->fulfill(promise1.release());\n EXPECT_FALSE(mockExecutor.empty());\n mockExecutor.runNext();\n ASSERT_EQ(result, 123);\n}\n\nTEST(PromiseTest, MoveSemantics) {\n MockExecutor mockExecutor;\n\n PromiseFulfiller>::Callback* fulfiller;\n auto promise = newPromise>>(&fulfiller);\n\n OwnedPtr ptr = newOwned(12);\n\n int result = 0;\n\n Promise promise2 = mockExecutor.when(promise, ptr)(\n [&result](OwnedPtr i, OwnedPtr j) {\n result = *i + *j;\n });\n\n fulfiller->fulfill(newOwned(34));\n mockExecutor.runNext();\n ASSERT_EQ(result, 46);\n}\n\nTEST(PromiseTest, Cancel) {\n MockExecutor mockExecutor;\n\n PromiseFulfiller::Callback* fulfiller;\n auto promise = newPromise>(&fulfiller);\n\n Promise promise2 = mockExecutor.when(promise)(\n [](int i) {\n ADD_FAILURE() << \"Can't get here.\";\n });\n\n EXPECT_TRUE(mockExecutor.empty());\n fulfiller->fulfill(5);\n EXPECT_FALSE(mockExecutor.empty());\n promise2.release();\n EXPECT_TRUE(mockExecutor.empty());\n}\n\nTEST(PromiseTest, VoidPromise) {\n MockExecutor mockExecutor;\n\n PromiseFulfiller::Callback* fulfiller;\n auto promise = newPromise>(&fulfiller);\n\n bool triggered = false;\n\n Promise promise2 = mockExecutor.when(promise)(\n [&triggered](Void) {\n triggered = true;\n });\n\n EXPECT_FALSE(triggered);\n fulfiller->fulfill();\n EXPECT_FALSE(triggered);\n mockExecutor.runNext();\n EXPECT_TRUE(triggered);\n}\n\nTEST(PromiseTest, Exception) {\n MockExecutor mockExecutor;\n\n PromiseFulfiller::Callback* fulfiller1;\n auto promise1 = newPromise>(&fulfiller1);\n PromiseFulfiller::Callback* fulfiller2;\n auto promise2 = newPromise>(&fulfiller2);\n\n bool triggered = false;\n\n Promise promise3 = mockExecutor.when(promise1, promise2, 123)(\n [](int i, int j, int k) {\n ADD_FAILURE() << \"Can't get here.\";\n }, [&triggered](MaybeException i, MaybeException j, int k) {\n triggered = true;\n\n ASSERT_TRUE(i.isException());\n ASSERT_FALSE(j.isException());\n ASSERT_EQ(123, k);\n ASSERT_EQ(456, j.get());\n\n try {\n i.get();\n ADD_FAILURE() << \"Expected exception.\";\n } catch (const std::logic_error& e) {\n ASSERT_STREQ(\"test\", e.what());\n }\n });\n\n try {\n throw std::logic_error(\"test\");\n } catch (...) {\n fulfiller1->propagateCurrentException();\n }\n fulfiller2->fulfill(456);\n\n EXPECT_FALSE(triggered);\n\n mockExecutor.runNext();\n\n EXPECT_TRUE(triggered);\n}\n\nTEST(PromiseTest, ExceptionInCallback) {\n MockExecutor mockExecutor;\n\n PromiseFulfiller::Callback* fulfiller1;\n auto promise1 = newPromise>(&fulfiller1);\n\n Promise promise2 = mockExecutor.when(promise1)(\n [](int a) -> int {\n throw std::logic_error(\"test\");\n });\n\n bool triggered = false;\n\n Promise promise3 = mockExecutor.when(promise2)(\n [](int i) {\n ADD_FAILURE() << \"Can't get here.\";\n }, [&triggered](MaybeException i) {\n triggered = true;\n\n ASSERT_TRUE(i.isException());\n try {\n i.get();\n ADD_FAILURE() << \"Expected exception.\";\n } catch (const std::logic_error& e) {\n ASSERT_STREQ(\"test\", e.what());\n }\n });\n\n EXPECT_TRUE(mockExecutor.empty());\n fulfiller1->fulfill(12);\n EXPECT_FALSE(mockExecutor.empty());\n mockExecutor.runNext();\n EXPECT_FALSE(mockExecutor.empty());\n mockExecutor.runNext();\n ASSERT_TRUE(triggered);\n}\n\nTEST(PromiseTest, ExceptionPropagation) {\n MockExecutor mockExecutor;\n\n PromiseFulfiller::Callback* fulfiller1;\n auto promise1 = newPromise>(&fulfiller1);\n\n Promise promise2 = mockExecutor.when(promise1)(\n [](int a) {\n ADD_FAILURE() << \"Can't get here.\";\n });\n\n bool triggered = false;\n\n Promise promise3 = mockExecutor.when(promise2)(\n [](Void) {\n ADD_FAILURE() << \"Can't get here.\";\n }, [&triggered](MaybeException i) {\n triggered = true;\n\n ASSERT_TRUE(i.isException());\n try {\n i.get();\n ADD_FAILURE() << \"Expected exception.\";\n } catch (const std::logic_error& e) {\n ASSERT_STREQ(\"test\", e.what());\n }\n });\n\n EXPECT_TRUE(mockExecutor.empty());\n try {\n throw std::logic_error(\"test\");\n } catch (...) {\n fulfiller1->propagateCurrentException();\n }\n EXPECT_FALSE(mockExecutor.empty());\n mockExecutor.runNext();\n EXPECT_FALSE(mockExecutor.empty());\n mockExecutor.runNext();\n ASSERT_TRUE(triggered);\n}\n\n} // namespace\n} // namespace ekam\n" }, { "path": "src/base/Table.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_BASE_TABLE_H_\n#define KENTONSCODE_BASE_TABLE_H_\n\n#include \n#include \n#include \n\nnamespace ekam {\n\n// =======================================================================================\n// Internal helpers. Please ignore.\n\ntemplate \nclass DummyMap {\npublic:\n typedef std::pair value_type;\n typedef value_type* iterator;\n typedef const value_type* const_iterator;\n inline iterator begin() { return NULL; }\n inline iterator end() { return NULL; }\n inline const_iterator begin() const { return NULL; }\n inline const_iterator end() const { return NULL; }\n inline iterator insert(const value_type& value) {\n return NULL;\n }\n inline int erase(const iterator& iter) {\n return 0;\n }\n inline void swap(DummyMap& other) {}\n};\n\ntemplate \nstruct ChooseType;\ntemplate \nstruct ChooseType : public Choices::Choice0 {};\ntemplate \nstruct ChooseType : public Choices::Choice1 {};\ntemplate \nstruct ChooseType : public Choices::Choice2 {};\n\n// =======================================================================================\n\ntemplate , typename Eq = std::equal_to >\nstruct IndexedColumn {\n typedef T Value;\n typedef std::unordered_multimap Index;\n};\n\ntemplate , typename Eq = std::equal_to >\nstruct UniqueColumn {\n typedef T Value;\n typedef std::unordered_map Index;\n};\n\ntemplate , typename Eq = std::equal_to >\nstruct Column {\n typedef T Value;\n typedef DummyMap Index;\n};\n\nstruct EmptyColumn {\n struct Value {};\n typedef DummyMap Index;\n};\n\ntemplate \nclass Table {\nprivate:\n struct Columns {\n#define CHOICE(INDEX) \\\n struct Choice##INDEX : public Column##INDEX { \\\n static inline typename Column##INDEX::Index* index(Table* table) { \\\n return &table->index##INDEX; \\\n } \\\n static inline const typename Column##INDEX::Index& index(const Table& table) { \\\n return table.index##INDEX; \\\n } \\\n template \\\n static inline const typename Column##INDEX::Value& cell(const Row& row) { \\\n return row.cell##INDEX; \\\n } \\\n };\n CHOICE(0)\n CHOICE(1)\n CHOICE(2)\n#undef CHOICE\n };\n template \n class Column : public ChooseType {};\n\npublic:\n Table() : deletedCount(0) {}\n ~Table() {}\n\n class Row {\n public:\n inline Row() {}\n\n template \n inline const typename Column::Value& cell() const {\n return Column::cell(*this);\n }\n\n private:\n typename Column<0>::Value cell0;\n typename Column<1>::Value cell1;\n typename Column<2>::Value cell2;\n bool deleted;\n\n inline Row(const typename Column<0>::Value& cell0,\n const typename Column<1>::Value& cell1,\n const typename Column<2>::Value& cell2)\n : cell0(cell0), cell1(cell1), cell2(cell2), deleted(false) {}\n\n friend class Table;\n };\n\n class RowIterator {\n public:\n inline RowIterator() {}\n RowIterator(const Table& table)\n : current(NULL), nextIter(table.rows.begin()), end(table.rows.end()) {}\n\n bool next() {\n while (true) {\n if (nextIter == end) {\n return false;\n } else if (!nextIter->deleted) {\n current = &*nextIter;\n ++nextIter;\n return true;\n }\n ++nextIter;\n }\n }\n\n template \n inline const typename Column::Value& cell() const {\n return Column::cell(*current);\n }\n\n private:\n const Row* current;\n typename std::vector::const_iterator nextIter;\n const typename std::vector::const_iterator end;\n };\n\n template \n class SearchIterator {\n public:\n inline SearchIterator() {}\n SearchIterator(const Table& table, const typename Column::Value& value)\n : table(table), current(NULL),\n range(Column::index(table).equal_range(value)) {}\n\n inline bool next() {\n while (true) {\n if (range.first == range.second) {\n return false;\n }\n\n const Row* row = &table.rows[range.first->second];\n\n if (!row->deleted) {\n current = row;\n ++range.first;\n return true;\n }\n\n ++range.first;\n }\n }\n\n template \n inline const typename Column::Value& cell() const {\n return Column::cell(*current);\n }\n\n private:\n typedef typename Column::Index::const_iterator InnerIter;\n const Table& table;\n const Row* current;\n std::pair range;\n };\n\n template \n const Row* find(const typename Column::Value& value) const {\n typename Column::Index::const_iterator iter =\n Column::index(*this).find(value);\n if (iter == Column::index(*this).end()) {\n return NULL;\n } else {\n const Row* row = &rows[iter->second];\n return row->deleted ? NULL : row;\n }\n }\n\n template \n const size_t erase(const typename Column::Value& value) {\n typedef typename Column::Index::iterator ColumnIterator;\n std::pair range =\n Column::index(this)->equal_range(value);\n\n size_t count = 0;\n for (ColumnIterator iter = range.first; iter != range.second; ++iter) {\n if (!rows[iter->second].deleted) {\n rows[iter->second].deleted = true;\n ++count;\n }\n }\n\n deletedCount += count;\n if (deletedCount >= 16 && deletedCount > rows.size() / 2) {\n refresh();\n } else {\n // This isn't strictly necessary, but since it's convenient, let's delete the range\n // from the index.\n Column::index(this)->erase(range.first, range.second);\n }\n\n return count;\n }\n\n void add(const typename Column<0>::Value& value0,\n const typename Column<1>::Value& value1 = EmptyColumn::Value(),\n const typename Column<2>::Value& value2 = EmptyColumn::Value()) {\n handleInsertResult(index0.insert(typename Column<0>::Index::value_type(value0, rows.size())));\n handleInsertResult(index1.insert(typename Column<1>::Index::value_type(value1, rows.size())));\n handleInsertResult(index2.insert(typename Column<2>::Index::value_type(value2, rows.size())));\n rows.push_back(Row(value0, value1, value2));\n }\n\n template \n const bool has(const typename Column::Value& value) {\n typedef typename Column::Index::iterator ColumnIterator;\n std::pair range =\n Column::index(this)->equal_range(value);\n\n for (ColumnIterator iter = range.first; iter != range.second; ++iter) {\n if (!rows[iter->second].deleted) {\n return true;\n }\n }\n return false;\n }\n\n int size() {\n return rows.size() - deletedCount;\n }\n int capacity() {\n return rows.capacity();\n }\n\n template \n int indexSize() {\n return Column::index(this)->size();\n }\n\nprivate:\n std::vector rows;\n size_t deletedCount;\n\n typename Column<0>::Index index0;\n typename Column<1>::Index index1;\n typename Column<2>::Index index2;\n\n void refresh() {\n std::vector newLocations;\n newLocations.reserve(rows.size());\n\n {\n std::vector newRows;\n newRows.reserve(rows.size() - deletedCount);\n\n for (size_t i = 0; i < rows.size(); i++) {\n if (rows[i].deleted) {\n newLocations.push_back(-1);\n } else {\n newLocations.push_back(newRows.size());\n newRows.push_back(rows[i]);\n }\n }\n\n rows.swap(newRows);\n deletedCount = 0;\n }\n\n refreshColumn<0>(newLocations);\n refreshColumn<1>(newLocations);\n refreshColumn<2>(newLocations);\n }\n\n template \n void refreshColumn(const std::vector& newLocations) {\n typedef Column C;\n typedef typename C::Index::iterator ColumnIterator;\n typename C::Index newIndex;\n for (ColumnIterator iter = C::index(this)->begin();\n iter != C::index(this)->end(); ++iter) {\n int newLocation = newLocations[iter->second];\n if (newLocation != -1) {\n newIndex.insert(std::make_pair(iter->first, newLocation));\n }\n }\n C::index(this)->swap(newIndex);\n }\n\n template \n void handleInsertResult(const std::pair& result) {\n if (!result.second) {\n rows[result.first->second].deleted = true;\n result.first->second = rows.size();\n }\n }\n\n template \n void handleInsertResult(const Iterator& iter) {}\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_BASE_TABLE_H_\n" }, { "path": "src/base/Table_test.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"Table.h\"\n#include \n#include \n#include \n#include \n#include \n\nnamespace ekam {\nnamespace {\n\n#define ASSERT(EXPRESSION) \\\n if (!(EXPRESSION)) { \\\n fprintf(stderr, \"%s:%d: FAILED: %s\\n\", __FILE__, __LINE__, #EXPRESSION); \\\n exit(1); \\\n }\n\nvoid testTable() {\n {\n typedef Table > MyTable;\n MyTable table;\n\n table.add(1234);\n table.add(5678);\n\n const MyTable::Row* row = table.find<0>(1234);\n ASSERT(table.find<0>(4321) == NULL);\n ASSERT(row != NULL);\n ASSERT(row->cell<0>() == 1234);\n row = table.find<0>(5678);\n ASSERT(row != NULL);\n ASSERT(row->cell<0>() == 5678);\n\n {\n std::multiset values;\n MyTable::RowIterator iter(table);\n\n ASSERT(iter.next());\n values.insert(iter.cell<0>());\n ASSERT(iter.next());\n values.insert(iter.cell<0>());\n ASSERT(!iter.next());\n\n ASSERT(values.count(1234) == 1);\n ASSERT(values.count(5678) == 1);\n }\n\n table.erase<0>(1234);\n\n ASSERT(table.find<0>(4321) == NULL);\n ASSERT(table.find<0>(1234) == NULL);\n row = table.find<0>(5678);\n ASSERT(row != NULL);\n ASSERT(row->cell<0>() == 5678);\n }\n\n {\n typedef Table, IndexedColumn > MyTable;\n MyTable table;\n\n table.add(12, 34);\n table.add(56, 34);\n\n {\n std::multiset values;\n MyTable::SearchIterator<1> iter(table, 34);\n\n ASSERT(iter.next());\n values.insert(iter.cell<0>());\n ASSERT(iter.next());\n values.insert(iter.cell<0>());\n ASSERT(!iter.next());\n\n ASSERT(values.count(12) == 1);\n ASSERT(values.count(56) == 1);\n }\n\n table.add(12, 78);\n const MyTable::Row* row = table.find<0>(12);\n ASSERT(row != NULL);\n ASSERT(row->cell<0>() == 12);\n ASSERT(row->cell<1>() == 78);\n\n {\n MyTable::SearchIterator<1> iter(table, 34);\n\n ASSERT(iter.next());\n ASSERT(iter.cell<0>() == 56);\n ASSERT(!iter.next());\n }\n }\n\n {\n typedef Table, IndexedColumn, Column > MyTable;\n MyTable table;\n\n table.add(\"foo\", 1, 'f');\n table.add(\"foo\", 2, 'o');\n table.add(\"foo\", 3, 'o');\n table.add(\"bar\", 1, 'b');\n table.add(\"bar\", 2, 'a');\n table.add(\"bar\", 3, 'r');\n\n {\n std::map values;\n MyTable::SearchIterator<0> iter(table, \"bar\");\n\n ASSERT(iter.next());\n ASSERT(iter.cell<0>() == \"bar\");\n values[iter.cell<1>()] = iter.cell<2>();\n ASSERT(iter.next());\n ASSERT(iter.cell<0>() == \"bar\");\n values[iter.cell<1>()] = iter.cell<2>();\n ASSERT(iter.next());\n ASSERT(iter.cell<0>() == \"bar\");\n values[iter.cell<1>()] = iter.cell<2>();\n ASSERT(!iter.next());\n\n ASSERT(values[1] == 'b');\n ASSERT(values[2] == 'a');\n ASSERT(values[3] == 'r');\n }\n\n {\n std::map values;\n MyTable::SearchIterator<1> iter(table, 2);\n\n ASSERT(iter.next());\n ASSERT(iter.cell<1>() == 2);\n values[iter.cell<0>()] = iter.cell<2>();\n ASSERT(iter.next());\n ASSERT(iter.cell<1>() == 2);\n values[iter.cell<0>()] = iter.cell<2>();\n ASSERT(!iter.next());\n\n ASSERT(values[\"foo\"] == 'o');\n ASSERT(values[\"bar\"] == 'a');\n }\n }\n\n {\n typedef Table, IndexedColumn > MyTable;\n MyTable table;\n\n for (int i = 0; i < 50; i++) {\n table.add(123, i);\n table.add(456, i);\n table.add(789, i);\n }\n\n ASSERT(table.size() == 150);\n ASSERT(table.capacity() >= 150);\n ASSERT(table.indexSize<0>() == 150);\n ASSERT(table.indexSize<1>() == 150);\n table.erase<0>(123);\n\n ASSERT(table.size() == 100);\n ASSERT(table.capacity() >= 150);\n ASSERT(table.indexSize<0>() == 100);\n ASSERT(table.indexSize<1>() == 150);\n table.erase<0>(456);\n\n ASSERT(table.size() == 50);\n ASSERT(table.capacity() == 50);\n ASSERT(table.indexSize<0>() == 50);\n ASSERT(table.indexSize<1>() == 50);\n\n const MyTable::Row* row = table.find<1>(5);\n ASSERT(row != NULL);\n ASSERT(row->cell<0>() == 789);\n\n {\n std::multiset values;\n MyTable::SearchIterator<0> iter(table, 789);\n\n for (int i = 0; i < 50; i++) {\n ASSERT(iter.next());\n values.insert(iter.cell<1>());\n }\n ASSERT(!iter.next());\n\n for (int i = 0; i < 50; i++) {\n ASSERT(values.count(i) == 1);\n }\n }\n\n ASSERT(!MyTable::SearchIterator<0>(table, 123).next());\n ASSERT(!MyTable::SearchIterator<0>(table, 456).next());\n\n {\n std::multiset values0;\n std::multiset values1;\n MyTable::RowIterator iter(table);\n\n for (int i = 0; i < 50; i++) {\n ASSERT(iter.next());\n values0.insert(iter.cell<0>());\n values1.insert(iter.cell<1>());\n }\n ASSERT(!iter.next());\n\n ASSERT(values0.count(789) == 50);\n for (int i = 0; i < 50; i++) {\n ASSERT(values1.count(i) == 1);\n }\n }\n }\n}\n\n} // namespace\n} // namespace ekam\n\nint main(int argc, char* argv[]) {\n ekam::testTable();\n return 0;\n}\n" }, { "path": "src/base/sha256.cpp", "content": "/********************************************************************\n * Pieces of endian.h from FreeBSD *\n ********************************************************************/\n/*-\n * Copyright (c) 2002 Thomas Moestl \n * All rights reserved.\n *\n * Redistribution and use in source and binary forms, with or without\n * modification, are permitted provided that the following conditions\n * are met:\n * 1. Redistributions of source code must retain the above copyright\n * notice, this list of conditions and the following disclaimer.\n * 2. Redistributions in binary form must reproduce the above copyright\n * notice, this list of conditions and the following disclaimer in the\n * documentation and/or other materials provided with the distribution.\n *\n * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND\n * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\n * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE\n * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE\n * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\n * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS\n * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)\n * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT\n * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY\n * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF\n * SUCH DAMAGE.\n *\n * $FreeBSD: src/sys/sys/endian.h,v 1.6.32.1 2009/08/03 08:13:06 kensmith Exp $\n */\n\n#include \n\ninline void\nbe32enc(void *pp, uint32_t u)\n{\n unsigned char *p = (unsigned char *)pp;\n\n p[0] = (u >> 24) & 0xff;\n p[1] = (u >> 16) & 0xff;\n p[2] = (u >> 8) & 0xff;\n p[3] = u & 0xff;\n}\n\ninline uint32_t\nbe32dec(const void *pp)\n{\n unsigned char const *p = (unsigned char const *)pp;\n\n return ((p[0] << 24) | (p[1] << 16) | (p[2] << 8) | p[3]);\n}\n\n/********************************************************************\n * sha256c.c from FreeBSD's libmd *\n * Modified to compile as C++ and placed in ekam namespace. *\n ********************************************************************/\n/*-\n * Copyright 2005 Colin Percival\n * All rights reserved.\n *\n * Redistribution and use in source and binary forms, with or without\n * modification, are permitted provided that the following conditions\n * are met:\n * 1. Redistributions of source code must retain the above copyright\n * notice, this list of conditions and the following disclaimer.\n * 2. Redistributions in binary form must reproduce the above copyright\n * notice, this list of conditions and the following disclaimer in the\n * documentation and/or other materials provided with the distribution.\n *\n * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND\n * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\n * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE\n * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE\n * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\n * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS\n * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)\n * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT\n * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY\n * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF\n * SUCH DAMAGE.\n */\n\n/*\n#include \n__FBSDID(\"$FreeBSD: src/lib/libmd/sha256c.c,v 1.2.10.1 2009/08/03 08:13:06 kensmith Exp $\");\n*/\n\n#include \n\n#include \n\n#include \"sha256.h\"\n\nnamespace ekam {\n\n#if BYTE_ORDER == BIG_ENDIAN\n\n/* Copy a vector of big-endian uint32_t into a vector of bytes */\n#define be32enc_vect(dst, src, len)\t\\\n\tmemcpy((void *)dst, (const void *)src, (size_t)len)\n\n/* Copy a vector of bytes into a vector of big-endian uint32_t */\n#define be32dec_vect(dst, src, len)\t\\\n\tmemcpy((void *)dst, (const void *)src, (size_t)len)\n\n#else /* BYTE_ORDER != BIG_ENDIAN */\n\n/*\n * Encode a length len/4 vector of (uint32_t) into a length len vector of\n * (unsigned char) in big-endian form. Assumes len is a multiple of 4.\n */\nstatic void\nbe32enc_vect(unsigned char *dst, const uint32_t *src, size_t len)\n{\n\tsize_t i;\n\n\tfor (i = 0; i < len / 4; i++)\n\t\tbe32enc(dst + i * 4, src[i]);\n}\n\n/*\n * Decode a big-endian length len vector of (unsigned char) into a length\n * len/4 vector of (uint32_t). Assumes len is a multiple of 4.\n */\nstatic void\nbe32dec_vect(uint32_t *dst, const unsigned char *src, size_t len)\n{\n\tsize_t i;\n\n\tfor (i = 0; i < len / 4; i++)\n\t\tdst[i] = be32dec(src + i * 4);\n}\n\n#endif /* BYTE_ORDER != BIG_ENDIAN */\n\n/* Elementary functions used by SHA256 */\n#define Ch(x, y, z)\t((x & (y ^ z)) ^ z)\n#define Maj(x, y, z)\t((x & (y | z)) | (y & z))\n#define SHR(x, n)\t(x >> n)\n#define ROTR(x, n)\t((x >> n) | (x << (32 - n)))\n#define S0(x)\t\t(ROTR(x, 2) ^ ROTR(x, 13) ^ ROTR(x, 22))\n#define S1(x)\t\t(ROTR(x, 6) ^ ROTR(x, 11) ^ ROTR(x, 25))\n#define s0(x)\t\t(ROTR(x, 7) ^ ROTR(x, 18) ^ SHR(x, 3))\n#define s1(x)\t\t(ROTR(x, 17) ^ ROTR(x, 19) ^ SHR(x, 10))\n\n/* SHA256 round function */\n#define RND(a, b, c, d, e, f, g, h, k)\t\t\t\\\n\tt0 = h + S1(e) + Ch(e, f, g) + k;\t\t\\\n\tt1 = S0(a) + Maj(a, b, c);\t\t\t\\\n\td += t0;\t\t\t\t\t\\\n\th = t0 + t1;\n\n/* Adjusted round function for rotating state */\n#define RNDr(S, W, i, k)\t\t\t\\\n\tRND(S[(64 - i) % 8], S[(65 - i) % 8],\t\\\n\t S[(66 - i) % 8], S[(67 - i) % 8],\t\\\n\t S[(68 - i) % 8], S[(69 - i) % 8],\t\\\n\t S[(70 - i) % 8], S[(71 - i) % 8],\t\\\n\t W[i] + k)\n\n/*\n * SHA256 block compression function. The 256-bit state is transformed via\n * the 512-bit input block to produce a new state.\n */\nstatic void\nSHA256_Transform(uint32_t * state, const unsigned char block[64])\n{\n\tuint32_t W[64];\n\tuint32_t S[8];\n\tuint32_t t0, t1;\n\tint i;\n\n\t/* 1. Prepare message schedule W. */\n\tbe32dec_vect(W, block, 64);\n\tfor (i = 16; i < 64; i++)\n\t\tW[i] = s1(W[i - 2]) + W[i - 7] + s0(W[i - 15]) + W[i - 16];\n\n\t/* 2. Initialize working variables. */\n\tmemcpy(S, state, 32);\n\n\t/* 3. Mix. */\n\tRNDr(S, W, 0, 0x428a2f98);\n\tRNDr(S, W, 1, 0x71374491);\n\tRNDr(S, W, 2, 0xb5c0fbcf);\n\tRNDr(S, W, 3, 0xe9b5dba5);\n\tRNDr(S, W, 4, 0x3956c25b);\n\tRNDr(S, W, 5, 0x59f111f1);\n\tRNDr(S, W, 6, 0x923f82a4);\n\tRNDr(S, W, 7, 0xab1c5ed5);\n\tRNDr(S, W, 8, 0xd807aa98);\n\tRNDr(S, W, 9, 0x12835b01);\n\tRNDr(S, W, 10, 0x243185be);\n\tRNDr(S, W, 11, 0x550c7dc3);\n\tRNDr(S, W, 12, 0x72be5d74);\n\tRNDr(S, W, 13, 0x80deb1fe);\n\tRNDr(S, W, 14, 0x9bdc06a7);\n\tRNDr(S, W, 15, 0xc19bf174);\n\tRNDr(S, W, 16, 0xe49b69c1);\n\tRNDr(S, W, 17, 0xefbe4786);\n\tRNDr(S, W, 18, 0x0fc19dc6);\n\tRNDr(S, W, 19, 0x240ca1cc);\n\tRNDr(S, W, 20, 0x2de92c6f);\n\tRNDr(S, W, 21, 0x4a7484aa);\n\tRNDr(S, W, 22, 0x5cb0a9dc);\n\tRNDr(S, W, 23, 0x76f988da);\n\tRNDr(S, W, 24, 0x983e5152);\n\tRNDr(S, W, 25, 0xa831c66d);\n\tRNDr(S, W, 26, 0xb00327c8);\n\tRNDr(S, W, 27, 0xbf597fc7);\n\tRNDr(S, W, 28, 0xc6e00bf3);\n\tRNDr(S, W, 29, 0xd5a79147);\n\tRNDr(S, W, 30, 0x06ca6351);\n\tRNDr(S, W, 31, 0x14292967);\n\tRNDr(S, W, 32, 0x27b70a85);\n\tRNDr(S, W, 33, 0x2e1b2138);\n\tRNDr(S, W, 34, 0x4d2c6dfc);\n\tRNDr(S, W, 35, 0x53380d13);\n\tRNDr(S, W, 36, 0x650a7354);\n\tRNDr(S, W, 37, 0x766a0abb);\n\tRNDr(S, W, 38, 0x81c2c92e);\n\tRNDr(S, W, 39, 0x92722c85);\n\tRNDr(S, W, 40, 0xa2bfe8a1);\n\tRNDr(S, W, 41, 0xa81a664b);\n\tRNDr(S, W, 42, 0xc24b8b70);\n\tRNDr(S, W, 43, 0xc76c51a3);\n\tRNDr(S, W, 44, 0xd192e819);\n\tRNDr(S, W, 45, 0xd6990624);\n\tRNDr(S, W, 46, 0xf40e3585);\n\tRNDr(S, W, 47, 0x106aa070);\n\tRNDr(S, W, 48, 0x19a4c116);\n\tRNDr(S, W, 49, 0x1e376c08);\n\tRNDr(S, W, 50, 0x2748774c);\n\tRNDr(S, W, 51, 0x34b0bcb5);\n\tRNDr(S, W, 52, 0x391c0cb3);\n\tRNDr(S, W, 53, 0x4ed8aa4a);\n\tRNDr(S, W, 54, 0x5b9cca4f);\n\tRNDr(S, W, 55, 0x682e6ff3);\n\tRNDr(S, W, 56, 0x748f82ee);\n\tRNDr(S, W, 57, 0x78a5636f);\n\tRNDr(S, W, 58, 0x84c87814);\n\tRNDr(S, W, 59, 0x8cc70208);\n\tRNDr(S, W, 60, 0x90befffa);\n\tRNDr(S, W, 61, 0xa4506ceb);\n\tRNDr(S, W, 62, 0xbef9a3f7);\n\tRNDr(S, W, 63, 0xc67178f2);\n\n\t/* 4. Mix local working variables into global state */\n\tfor (i = 0; i < 8; i++)\n\t\tstate[i] += S[i];\n}\n\nstatic unsigned char PAD[64] = {\n\t0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n\t0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n\t0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n\t0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0\n};\n\n/* Add padding and terminating bit-count. */\nstatic void\nSHA256_Pad(SHA256_CTX * ctx)\n{\n\tunsigned char len[8];\n\tuint32_t r, plen;\n\n\t/*\n\t * Convert length to a vector of bytes -- we do this now rather\n\t * than later because the length will change after we pad.\n\t */\n\tbe32enc_vect(len, ctx->count, 8);\n\n\t/* Add 1--64 bytes so that the resulting length is 56 mod 64 */\n\tr = (ctx->count[1] >> 3) & 0x3f;\n\tplen = (r < 56) ? (56 - r) : (120 - r);\n\tSHA256_Update(ctx, PAD, (size_t)plen);\n\n\t/* Add the terminating bit-count */\n\tSHA256_Update(ctx, len, 8);\n}\n\n/* SHA-256 initialization. Begins a SHA-256 operation. */\nvoid\nSHA256_Init(SHA256_CTX * ctx)\n{\n\n\t/* Zero bits processed so far */\n\tctx->count[0] = ctx->count[1] = 0;\n\n\t/* Magic initialization constants */\n\tctx->state[0] = 0x6A09E667;\n\tctx->state[1] = 0xBB67AE85;\n\tctx->state[2] = 0x3C6EF372;\n\tctx->state[3] = 0xA54FF53A;\n\tctx->state[4] = 0x510E527F;\n\tctx->state[5] = 0x9B05688C;\n\tctx->state[6] = 0x1F83D9AB;\n\tctx->state[7] = 0x5BE0CD19;\n}\n\n/* Add bytes into the hash */\nvoid\nSHA256_Update(SHA256_CTX * ctx, const void *in, size_t len)\n{\n\tuint32_t bitlen[2];\n\tuint32_t r;\n\tconst unsigned char *src = reinterpret_cast(in);\n\n\t/* Number of bytes left in the buffer from previous updates */\n\tr = (ctx->count[1] >> 3) & 0x3f;\n\n\t/* Convert the length into a number of bits */\n\tbitlen[1] = ((uint32_t)len) << 3;\n\tbitlen[0] = (uint32_t)(len >> 29);\n\n\t/* Update number of bits */\n\tif ((ctx->count[1] += bitlen[1]) < bitlen[1])\n\t\tctx->count[0]++;\n\tctx->count[0] += bitlen[0];\n\n\t/* Handle the case where we don't need to perform any transforms */\n\tif (len < 64 - r) {\n\t\tmemcpy(&ctx->buf[r], src, len);\n\t\treturn;\n\t}\n\n\t/* Finish the current block */\n\tmemcpy(&ctx->buf[r], src, 64 - r);\n\tSHA256_Transform(ctx->state, ctx->buf);\n\tsrc += 64 - r;\n\tlen -= 64 - r;\n\n\t/* Perform complete blocks */\n\twhile (len >= 64) {\n\t\tSHA256_Transform(ctx->state, src);\n\t\tsrc += 64;\n\t\tlen -= 64;\n\t}\n\n\t/* Copy left over data into buffer */\n\tmemcpy(ctx->buf, src, len);\n}\n\n/*\n * SHA-256 finalization. Pads the input data, exports the hash value,\n * and clears the context state.\n */\nvoid\nSHA256_Final(unsigned char digest[32], SHA256_CTX * ctx)\n{\n\n\t/* Add padding */\n\tSHA256_Pad(ctx);\n\n\t/* Write the hash */\n\tbe32enc_vect(digest, ctx->state, 32);\n\n\t/* Clear the context state */\n\tmemset((void *)ctx, 0, sizeof(*ctx));\n}\n\n} // namespace ekam\n" }, { "path": "src/base/sha256.h", "content": "/********************************************************************\n * sha256.h from FreeBSD's libmd *\n * Modified to compile as C++ and placed in ekam namespace. *\n ********************************************************************/\n/*-\n * Copyright 2005 Colin Percival\n * All rights reserved.\n *\n * Redistribution and use in source and binary forms, with or without\n * modification, are permitted provided that the following conditions\n * are met:\n * 1. Redistributions of source code must retain the above copyright\n * notice, this list of conditions and the following disclaimer.\n * 2. Redistributions in binary form must reproduce the above copyright\n * notice, this list of conditions and the following disclaimer in the\n * documentation and/or other materials provided with the distribution.\n *\n * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND\n * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\n * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE\n * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE\n * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\n * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS\n * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)\n * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT\n * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY\n * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF\n * SUCH DAMAGE.\n *\n * $FreeBSD: src/lib/libmd/sha256.h,v 1.2.10.1 2009/08/03 08:13:06 kensmith Exp $\n */\n\n#ifndef _SHA256_H_\n#define _SHA256_H_\n\n#include \n\nnamespace ekam {\n\ntypedef struct SHA256Context {\n\tuint32_t state[8];\n\tuint32_t count[2];\n\tunsigned char buf[64];\n} SHA256_CTX;\n\nvoid\tSHA256_Init(SHA256_CTX *);\nvoid\tSHA256_Update(SHA256_CTX *, const void *, size_t);\nvoid\tSHA256_Final(unsigned char [32], SHA256_CTX *);\nchar *SHA256_End(SHA256_CTX *, char *);\nchar *SHA256_File(const char *, char *);\nchar *SHA256_FileChunk(const char *, char *, off_t, off_t);\nchar *SHA256_Data(const void *, unsigned int, char *);\n\n} // namespace ekam\n\n#endif /* !_SHA256_H_ */\n" }, { "path": "src/ekam/Action.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"Action.h\"\n\nnamespace ekam {\n\nBuildContext::~BuildContext() noexcept(false) {}\nAction::~Action() {}\nActionFactory::~ActionFactory() {}\n\nconst int BuildContext::INSTALL_LOCATION_COUNT;\nconst char* const BuildContext::INSTALL_LOCATION_NAMES[INSTALL_LOCATION_COUNT] = {\n \"bin\", \"lib\", \"node_modules\"\n};\n\n} // namespace ekam\n" }, { "path": "src/ekam/Action.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_EKAM_ACTION_H_\n#define KENTONSCODE_EKAM_ACTION_H_\n\n#include \n#include \n#include \n\n#include \"base/OwnedPtr.h\"\n#include \"os/File.h\"\n#include \"Tag.h\"\n#include \"os/EventManager.h\"\n\nnamespace ekam {\n\nclass ActionFactory;\n\nclass ProcessExitCallback {\npublic:\n virtual ~ProcessExitCallback();\n\n // Negative = signal number.\n virtual void done(int exit_status) = 0;\n};\n\nclass BuildContext {\npublic:\n virtual ~BuildContext() noexcept(false);\n\n virtual File* findProvider(Tag id) = 0;\n virtual File* findInput(const std::string& path) = 0;\n\n enum InstallLocation {\n BIN,\n LIB,\n NODE_MODULES\n };\n static const int INSTALL_LOCATION_COUNT = 3;\n\n static const char* const INSTALL_LOCATION_NAMES[INSTALL_LOCATION_COUNT];\n\n virtual void provide(File* file, const std::vector& tags) = 0;\n virtual void install(File* file, InstallLocation location, const std::string& name) = 0;\n virtual void log(const std::string& text) = 0;\n\n virtual OwnedPtr newOutput(const std::string& path) = 0;\n\n virtual void addActionType(OwnedPtr factory) = 0;\n\n virtual void passed() = 0;\n virtual void failed() = 0;\n};\n\nclass Action {\npublic:\n virtual ~Action();\n\n virtual bool isSilent() { return false; }\n virtual std::string getVerb() = 0;\n virtual Promise start(EventManager* eventManager, BuildContext* context) = 0;\n};\n\nclass ActionFactory {\npublic:\n virtual ~ActionFactory();\n\n virtual void enumerateTriggerTags(std::back_insert_iterator > iter) = 0;\n virtual OwnedPtr tryMakeAction(const Tag& id, File* file) = 0;\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_EKAM_ACTION_H_\n" }, { "path": "src/ekam/ActionUtil.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"ActionUtil.h\"\n#include \n\nnamespace ekam {\n\nLogger::Logger(BuildContext* context, OwnedPtr stream)\n : context(context), stream(stream.release()) {}\nLogger::~Logger() {}\n\nPromise Logger::run(EventManager* eventManager) {\n return eventManager->when(stream->readAsync(eventManager, buffer, sizeof(buffer)))(\n [=](size_t size) -> Promise {\n if (size == 0) {\n return newFulfilledPromise();\n }\n context->log(std::string(buffer, size));\n return run(eventManager);\n }, [=](MaybeException error) {\n try {\n error.get();\n } catch (const std::exception& e) {\n context->log(e.what());\n context->failed();\n throw;\n } catch (...) {\n context->log(\"unknown exception\");\n context->failed();\n throw;\n }\n });\n}\n\n// =======================================================================================\n\nLineReader::LineReader(ByteStream* stream) : stream(stream) {}\nLineReader::~LineReader() {}\n\nPromise> LineReader::readLine(EventManager* eventManager) {\n std::string::size_type endpos = leftover.find_first_of('\\n');\n if (endpos != std::string::npos) {\n auto result = newOwned(leftover, 0, endpos);\n leftover.erase(0, endpos + 1);\n return newFulfilledPromise(result.release());\n }\n\n return eventManager->when(stream->readAsync(eventManager, buffer, sizeof(buffer)))(\n [=](size_t size) -> Promise> {\n if (size == 0) {\n if (leftover.empty()) {\n // No more data.\n return newFulfilledPromise(OwnedPtr(nullptr));\n } else {\n // Still have a line of text that had no trailing newline.\n auto line = newOwned(std::move(leftover));\n leftover.clear();\n return newFulfilledPromise(line.release());\n }\n }\n\n leftover.append(buffer, size);\n return readLine(eventManager);\n });\n}\n\n} // namespace ekam\n" }, { "path": "src/ekam/ActionUtil.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_EKAM_ACTIONUTIL_H_\n#define KENTONSCODE_EKAM_ACTIONUTIL_H_\n\n#include \n\n#include \"os/ByteStream.h\"\n#include \"Action.h\"\n\nnamespace ekam {\n\nclass Logger {\npublic:\n Logger(BuildContext* context, OwnedPtr stream);\n ~Logger();\n\n Promise run(EventManager* eventManager);\n\nprivate:\n class ReadAllFulfiller;\n\n BuildContext* context;\n OwnedPtr stream;\n char buffer[4096];\n};\n\nclass LineReader {\npublic:\n LineReader(ByteStream* stream);\n ~LineReader();\n\n Promise> readLine(EventManager* eventManager);\n\nprivate:\n ByteStream* stream;\n std::string leftover;\n char buffer[4096];\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_EKAM_ACTIONUTIL_H_\n" }, { "path": "src/ekam/ConsoleDashboard.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"ConsoleDashboard.h\"\n\n#include \n#include \n#include \n#include \n\n#include \"base/Debug.h\"\n\nnamespace ekam {\n\nclass ConsoleDashboard::LogFormatter {\npublic:\n LogFormatter(const std::string& text)\n : text(text.data()), end(text.data() + text.size()), wrapped(false) {\n eatWhitespace();\n }\n\n inline bool atEnd() {\n return text == end;\n }\n\n std::string getLine(int startColumn, int windowWidth) {\n std::string result;\n int column = startColumn;\n result.reserve((windowWidth - column) * 2);\n\n if (wrapped) {\n result.append(\" \");\n column += 2;\n wrapped = false;\n }\n\n while (text < end && *text != '\\n' && column < windowWidth) {\n if (isalnum(*text)) {\n const char* wordEnd = text;\n do {\n ++wordEnd;\n } while (wordEnd < end && isalnum(*wordEnd));\n\n int length = wordEnd - text;\n\n if (column + length <= windowWidth) {\n // Word fits on this line.\n bool isColored = false;\n if (strncasecmp(text, \"error\", length) == 0 ||\n strncasecmp(text, \"fail\", length) == 0 ||\n strncasecmp(text, \"failed\", length) == 0 ||\n strncasecmp(text, \"fatal\", length) == 0) {\n result.append(ANSI_COLOR_CODES[RED]);\n isColored = true;\n } else if (strncasecmp(text, \"warning\", length) == 0) {\n result.append(ANSI_COLOR_CODES[YELLOW]);\n isColored = true;\n }\n\n result.append(text, wordEnd);\n text = wordEnd;\n column += length;\n\n if (isColored) {\n result.append(ANSI_CLEAR_COLOR);\n }\n } else if (length >= 20 || column == startColumn) {\n // Really long word. Break it.\n int spaceAvailable = windowWidth - column;\n result.append(text, spaceAvailable);\n text += spaceAvailable;\n column = windowWidth;\n } else {\n // Word doesn't fit in remaining space. End the line.\n break;\n }\n } else {\n switch (*text) {\n case '\\t':\n column = (column & ~0x7) + 8;\n result.push_back(*text);\n break;\n case '\\033': // escape\n // ignore -- could be harmful\n // TODO: Parse and remove the subsequent terminal instruction to make the output not\n // suck. Or better yet, parse the sequence and interpret it. If it is an SGR code\n // (e.g. color), pass it through, and just make sure to reset later.\n break;\n default:\n if (*text >= '\\0' && *text < ' ') {\n // Ignore control character or weird whitespace.\n } else {\n result.push_back(*text);\n ++column;\n }\n break;\n }\n ++text;\n }\n }\n\n wrapped = !eatWhitespace();\n\n return result;\n }\n\nprivate:\n const char* text;\n const char* end;\n bool wrapped;\n\n // Eat whitespace up to and including a newline. Returns true if a newline was eaten.\n bool eatWhitespace() {\n while (text < end && isspace(*text) && *text != '\\n') {\n ++text;\n }\n if (text < end && *text == '\\n') {\n ++text;\n return true;\n } else {\n return false;\n }\n }\n\n bool tryHighlight(const char* word, Color color, int windowWidth,\n int* column, std::string* output) {\n int length = strlen(word);\n\n if (windowWidth - *column < length) {\n // Don't highlight words broken at line wrapping.\n return false;\n }\n\n if (end - text < length) {\n // Not enough characters left to match word.\n return false;\n }\n\n if (strncasecmp(text, word, length) != 0) {\n // Does not match.\n return false;\n }\n\n if (end - text > length && isalpha(text[length])) {\n // Character after word is a letter. Don't highlight.\n return false;\n }\n\n if (!output->empty() && isalpha((*output)[output->size() - 1])) {\n // Character before word was a letter. Don't highlight.\n return false;\n }\n\n output->append(ANSI_COLOR_CODES[color]);\n output->append(text, text + length);\n output->append(ANSI_CLEAR_COLOR);\n text += length;\n *column += length;\n return true;\n }\n};\n\nclass ConsoleDashboard::TaskImpl : public Dashboard::Task {\npublic:\n TaskImpl(ConsoleDashboard* dashboard, const std::string& verb,\n const std::string& noun, Silence silence);\n ~TaskImpl();\n\n // implements Task ---------------------------------------------------------------------\n void setState(TaskState state);\n void addOutput(const std::string& text);\n\nprivate:\n ConsoleDashboard* dashboard;\n TaskState state;\n Silence silence;\n std::string verb;\n std::string noun;\n std::string outputText;\n\n void removeFromRunning();\n void writeFinalLog(Color verbColor, const char* icon);\n\n friend class ConsoleDashboard;\n};\n\nConsoleDashboard::TaskImpl::TaskImpl(ConsoleDashboard* dashboard,\n const std::string& verb, const std::string& noun,\n Silence silence)\n : dashboard(dashboard), state(PENDING), silence(silence), verb(verb), noun(noun) {}\nConsoleDashboard::TaskImpl::~TaskImpl() {\n if (state == RUNNING) {\n dashboard->clearRunning();\n removeFromRunning();\n dashboard->drawRunning();\n }\n}\n\nvoid ConsoleDashboard::TaskImpl::setState(TaskState state) {\n // If state was previously BLOCKED, and we managed to un-block, then we don't care about the\n // reason why we were blocked, so clear the text.\n if (this->state == BLOCKED && (state == PENDING || state == RUNNING)) {\n outputText.clear();\n }\n\n if (this->state == RUNNING) {\n if (silence != SILENT) removeFromRunning();\n }\n\n this->state = state;\n\n dashboard->clearRunning();\n\n switch (state) {\n case PENDING:\n // Don't display.\n break;\n case RUNNING:\n if (silence != SILENT) dashboard->runningTasks.push_back(this);\n break;\n case DONE:\n writeFinalLog(DONE_COLOR, \" \");\n break;\n case PASSED:\n writeFinalLog(PASSED_COLOR, \"✔\");\n break;\n case FAILED:\n writeFinalLog(FAILED_COLOR, \"✘\");\n break;\n case BLOCKED:\n // Don't display.\n break;\n }\n\n dashboard->drawRunning();\n}\n\nvoid ConsoleDashboard::TaskImpl::addOutput(const std::string& text) {\n outputText.append(text);\n}\n\nvoid ConsoleDashboard::TaskImpl::removeFromRunning() {\n for (std::vector::iterator iter = dashboard->runningTasks.begin();\n iter != dashboard->runningTasks.end(); ++iter) {\n if (*iter == this) {\n dashboard->runningTasks.erase(iter);\n break;\n }\n }\n}\n\nvoid ConsoleDashboard::TaskImpl::writeFinalLog(Color verbColor, const char* icon) {\n // Silent tasks should not be written to the log, unless they had error messages.\n if (silence != SILENT || !outputText.empty()) {\n fprintf(dashboard->out, \"%s%s %s:%s %s\\n\",\n ANSI_COLOR_CODES[verbColor], icon, verb.c_str(), ANSI_CLEAR_COLOR, noun.c_str());\n\n // Write any output we have buffered.\n if (!outputText.empty()) {\n LogFormatter formatter(outputText);\n struct winsize windowSize;\n ioctl(dashboard->fd, TIOCGWINSZ, &windowSize);\n\n for (int i = 0; i < dashboard->maxDisplayedLogLines && !formatter.atEnd(); i++) {\n std::string line = formatter.getLine(4, windowSize.ws_col);\n fprintf(dashboard->out, \" %s\\n\", line.c_str());\n }\n\n if (!formatter.atEnd()) {\n fprintf(dashboard->out, \" ...(log truncated; use -l to increase log limit)...\\n\");\n }\n\n outputText.clear();\n }\n }\n}\n\n// =======================================================================================\n\nconst char* const ConsoleDashboard::ANSI_COLOR_CODES[] = {\n \"\\033[30m\",\n \"\\033[31m\",\n \"\\033[32m\",\n \"\\033[33m\",\n \"\\033[34m\",\n \"\\033[35m\",\n \"\\033[36m\",\n \"\\033[37m\",\n \"\\033[1;30m\",\n \"\\033[1;31m\",\n \"\\033[1;32m\",\n \"\\033[1;33m\",\n \"\\033[1;34m\",\n \"\\033[1;35m\",\n \"\\033[1;36m\",\n \"\\033[1;37m\"\n};\n\nconst char* const ConsoleDashboard::ANSI_CLEAR_COLOR = \"\\033[0m\";\n\n// Note: \\033[%dF (move cursor up %d lines and to beginning of line) doesn't work on some\n// terminals. \\033[%dA (move cursor up %d lines) does appear to work, so tack \\r on to that\n// to go to the beginning of the line.\nconst char* const ConsoleDashboard::ANSI_MOVE_CURSOR_UP = \"\\033[%dA\\r\";\nconst char* const ConsoleDashboard::ANSI_CLEAR_BELOW_CURSOR = \"\\033[0J\";\n\nconst ConsoleDashboard::Color ConsoleDashboard::DONE_COLOR = BRIGHT_BLUE;\nconst ConsoleDashboard::Color ConsoleDashboard::PASSED_COLOR = BRIGHT_GREEN;\nconst ConsoleDashboard::Color ConsoleDashboard::FAILED_COLOR = BRIGHT_RED;\nconst ConsoleDashboard::Color ConsoleDashboard::RUNNING_COLOR = BRIGHT_FUCHSIA;\n\nConsoleDashboard::ConsoleDashboard(FILE* output, int maxDisplayedLogLines)\n : fd(fileno(output)), out(output), maxDisplayedLogLines(maxDisplayedLogLines),\n runningTasksLineCount(0), lastDebugMessageCount(DebugMessage::getMessageCount()) {}\nConsoleDashboard::~ConsoleDashboard() {}\n\nOwnedPtr ConsoleDashboard::beginTask(\n const std::string& verb, const std::string& noun, Silence silence) {\n return newOwned(this, verb, noun, silence);\n}\n\nvoid ConsoleDashboard::clearRunning() {\n if (lastDebugMessageCount != DebugMessage::getMessageCount()) {\n // Some debug messages were printed. We don't want to clobber them. So we can't clear.\n return;\n }\n\n if (runningTasksLineCount > 0) {\n fprintf(out, ANSI_MOVE_CURSOR_UP, runningTasksLineCount);\n fputs(ANSI_CLEAR_BELOW_CURSOR, out);\n }\n}\n\nvoid ConsoleDashboard::drawRunning() {\n struct winsize windowSize;\n ioctl(fd, TIOCGWINSZ, &windowSize);\n\n // Leave a few lines for completed tasks.\n int spaceForTasks = windowSize.ws_row - 4;\n\n bool allTasksShown = static_cast(runningTasks.size()) < spaceForTasks;\n int displayCount = allTasksShown ? runningTasks.size() : spaceForTasks - 1;\n runningTasksLineCount = allTasksShown ? runningTasks.size() : spaceForTasks;\n\n for (int i = 0; i < displayCount; i++) {\n TaskImpl* task = runningTasks[i];\n int spaceForNoun = windowSize.ws_col - task->verb.size() - 2;\n\n fprintf(out, \"%s↺ %s:%s \", ANSI_COLOR_CODES[RUNNING_COLOR],\n task->verb.c_str(), ANSI_CLEAR_COLOR);\n\n if (static_cast(task->noun.size()) > spaceForNoun) {\n std::string shortenedNoun = task->noun.substr(task->noun.size() - spaceForNoun + 3);\n fprintf(out, \"...%s\", shortenedNoun.c_str());\n } else {\n fputs(task->noun.c_str(), out);\n }\n\n putc('\\n', out);\n }\n\n if (!allTasksShown) {\n fputs(\"...(more)...\\n\", out);\n }\n\n fflush(out);\n\n lastDebugMessageCount = DebugMessage::getMessageCount();\n}\n\n} // namespace ekam\n" }, { "path": "src/ekam/ConsoleDashboard.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_EKAM_CONSOLEDASHBOARD_H_\n#define KENTONSCODE_EKAM_CONSOLEDASHBOARD_H_\n\n#include \n#include \n#include \"Dashboard.h\"\n\nnamespace ekam {\n\nclass ConsoleDashboard : public Dashboard {\npublic:\n ConsoleDashboard(FILE* output, int maxDisplayedLogLines);\n ~ConsoleDashboard();\n\n // implements Dashboard ----------------------------------------------------------------\n OwnedPtr beginTask(const std::string& verb, const std::string& noun, Silence silence);\n\nprivate:\n class TaskImpl;\n class LogFormatter;\n\n int fd;\n FILE* out;\n int maxDisplayedLogLines;\n\n std::vector runningTasks;\n int runningTasksLineCount;\n int lastDebugMessageCount;\n\n enum Color {\n BLACK,\n RED,\n GREEN,\n YELLOW,\n BLUE,\n FUCHSIA,\n CYAN,\n WHITE,\n GRAY,\n BRIGHT_RED,\n BRIGHT_GREEN,\n BRIGHT_YELLOW,\n BRIGHT_BLUE,\n BRIGHT_FUCHSIA,\n BRIGHT_CYAN,\n BRIGHT_WHITE,\n };\n\n static const char* const ANSI_COLOR_CODES[];\n static const char* const ANSI_CLEAR_COLOR;\n static const char* const ANSI_MOVE_CURSOR_UP;\n static const char* const ANSI_CLEAR_BELOW_CURSOR;\n\n static const Color DONE_COLOR;\n static const Color PASSED_COLOR;\n static const Color FAILED_COLOR;\n static const Color RUNNING_COLOR;\n\n void clearRunning();\n void drawRunning();\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_EKAM_CONSOLEDASHBOARD_H_\n" }, { "path": "src/ekam/CppActionFactory.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"CppActionFactory.h\"\n\n#include \n#include \n\n#include \"base/Debug.h\"\n#include \"os/ByteStream.h\"\n#include \"os/Subprocess.h\"\n#include \"ActionUtil.h\"\n\nnamespace ekam {\n\n// compile action: produces object file, tags for all symbols declared therein.\n// (Compile action is now implemented in compile.ekam-rule.)\n// link action: triggers on \"main\" tag.\n\nnamespace {\n\nOwnedPtr getDepsFile(File* objectFile) {\n return objectFile->parent()->relative(objectFile->basename() + \".deps\");\n}\n\nbool isTestName(const std::string& name) {\n std::string::size_type pos = name.find_last_of(\"_-\");\n if (pos == std::string::npos) {\n return false;\n } else {\n std::string suffix = name.substr(pos + 1);\n return suffix == \"test\" || suffix == \"unittest\" || suffix == \"regtest\";\n }\n}\n\n} // namespace\n\nclass LinkAction : public Action {\npublic:\n enum Mode {\n NORMAL,\n GTEST,\n KJTEST,\n NODEJS,\n LIBFUZZER\n };\n\n LinkAction(File* file, Mode mode);\n ~LinkAction();\n\n // implements Action -------------------------------------------------------------------\n std::string getVerb();\n Promise start(EventManager* eventManager, BuildContext* context);\n\nprivate:\n class DepsSet {\n public:\n DepsSet() {}\n ~DepsSet() {}\n\n void addObject(BuildContext* context, File* objectFile);\n void enumerate(OwnedPtrVector::Appender output) {\n deps.releaseAll(output);\n }\n\n private:\n OwnedPtrMap deps;\n };\n\n static const Tag GTEST_MAIN;\n static const Tag KJTEST_MAIN;\n static const Tag TEST_EXECUTABLE;\n\n OwnedPtr file;\n Mode mode;\n\n Promise startTarget(EventManager* eventManager, BuildContext* context,\n const std::string& base, OwnedPtrVector& flatDeps,\n const std::string& target);\n};\n\nconst Tag LinkAction::GTEST_MAIN = Tag::fromName(\"gtest:main\");\nconst Tag LinkAction::KJTEST_MAIN = Tag::fromName(\"kjtest:main\");\nconst Tag LinkAction::TEST_EXECUTABLE = Tag::fromName(\"test:executable\");\n\nLinkAction::LinkAction(File* file, Mode mode) : file(file->clone()), mode(mode) {}\n\nLinkAction::~LinkAction() {}\n\nstd::string LinkAction::getVerb() {\n return \"link\";\n}\n\nvoid LinkAction::DepsSet::addObject(BuildContext* context, File* objectFile) {\n if (deps.contains(objectFile)) {\n return;\n }\n\n OwnedPtr ptr = objectFile->clone();\n File* rawptr = ptr.get(); // cannot inline due to undefined evaluation order\n deps.add(rawptr, ptr.release());\n\n OwnedPtr depsFile = getDepsFile(objectFile);\n if (depsFile->exists()) {\n std::string data = depsFile->readAll();\n\n std::string::size_type prevPos = 0;\n std::string::size_type pos = data.find_first_of('\\n');\n\n while (pos != std::string::npos) {\n std::string symbolName(data, prevPos, pos - prevPos);\n\n File* file = context->findProvider(Tag::fromName(\"c++symbol:\" + symbolName));\n if (file != NULL) {\n addObject(context, file);\n }\n\n prevPos = pos + 1;\n pos = data.find_first_of('\\n', prevPos);\n }\n }\n}\n\n// ---------------------------------------------------------------------------------------\n\nPromise LinkAction::start(EventManager* eventManager, BuildContext* context) {\n DepsSet deps;\n\n if (mode == GTEST) {\n File* gtestMain = context->findProvider(GTEST_MAIN);\n if (gtestMain == NULL) {\n context->log(\"Cannot find gtest_main.o.\");\n context->failed();\n return newFulfilledPromise();\n }\n\n deps.addObject(context, gtestMain);\n } else if (mode == KJTEST) {\n File* kjtestMain = context->findProvider(KJTEST_MAIN);\n if (kjtestMain == NULL) {\n context->log(\"Cannot find kj/test.o.\");\n context->failed();\n return newFulfilledPromise();\n }\n\n deps.addObject(context, kjtestMain);\n }\n\n deps.addObject(context, file.get());\n\n OwnedPtrVector flatDeps;\n deps.enumerate(flatDeps.appender());\n\n std::string base, ext;\n splitExtension(file->canonicalName(), &base, &ext);\n\n if (mode == NODEJS) {\n base += \".node\";\n }\n\n auto promise = startTarget(eventManager, context, base, flatDeps, \"\");\n\n const char* targets = getenv(\"CROSS_TARGETS\");\n if (targets != NULL) {\n auto addTarget = [&](std::string target) {\n OwnedPtrVector targetDeps;\n for (int i = 0; i < flatDeps.size(); i++) {\n std::string name, ext;\n splitExtension(flatDeps.get(i)->basename(), &name, &ext);\n targetDeps.add(flatDeps.get(i)->parent()->relative(name + '.' + target + ext));\n }\n\n promise = eventManager->when(std::move(promise))(\n [this, target = std::move(target), targetDeps = std::move(targetDeps),\n eventManager, context, base](Void) mutable {\n return startTarget(eventManager, context, base, targetDeps, target);\n });\n };\n\n while (const char* spacepos = strchr(targets, ' ')) {\n addTarget(std::string(targets, spacepos));\n targets = spacepos + 1;\n }\n addTarget(targets);\n }\n\n return promise;\n}\n\nPromise LinkAction::startTarget(\n EventManager* eventManager, BuildContext* context,\n const std::string& base, OwnedPtrVector& flatDeps,\n const std::string& target) {\n const char* cxx = getenv(\"CXX\");\n\n auto subprocess = newOwned();\n\n std::string compiler = cxx == NULL ? \"c++\" : cxx;\n\n auto slashpos = compiler.find_last_of('/');\n std::string compilerName = slashpos < 0 ? compiler : compiler.substr(slashpos + 1);\n\n if (target.empty()) {\n subprocess->addArgument(std::move(compiler));\n } else if (strstr(compilerName.c_str(), \"clang\")) {\n subprocess->addArgument(std::move(compiler));\n subprocess->addArgument(\"-target\");\n subprocess->addArgument(target);\n } else {\n subprocess->addArgument(target + '-' + compiler);\n }\n\n if (mode == NODEJS) {\n subprocess->addArgument(\"-shared\");\n } else if (context->findProvider(Tag::fromName(\"canonical:\" + base + \".link-static\")) !=\n nullptr) {\n subprocess->addArgument(\"-static\");\n }\n\n if (mode == LIBFUZZER) {\n const auto arg = getenv(\"EKAM_LIBFUZZER_LINKER_ARG\");\n subprocess->addArgument(arg == nullptr ? \"-fsanitize=fuzzer\" : arg);\n }\n\n subprocess->addArgument(\"-o\");\n\n auto executableFile = context->newOutput(target.empty() ? base : (base + \".\" + target));\n subprocess->addArgument(executableFile.get(), File::WRITE);\n\n if (isTestName(base)) {\n std::vector tags;\n tags.push_back(TEST_EXECUTABLE);\n context->provide(executableFile.get(), tags);\n }\n\n for (int i = 0; i < flatDeps.size(); i++) {\n subprocess->addArgument(flatDeps.get(i), File::READ);\n }\n\n const char* libs = nullptr;\n if (!target.empty()) {\n // Look for arch-specific libs.\n std::string targetLibsName = \"LIBS_\" + target;\n for (char& c: targetLibsName) {\n if (c == '-') c = '_';\n }\n libs = getenv(targetLibsName.c_str());\n }\n\n if (libs == nullptr) {\n libs = getenv(\"LIBS\");\n }\n\n if (libs != NULL) {\n while (const char* spacepos = strchr(libs, ' ')) {\n subprocess->addArgument(std::string(libs, spacepos));\n libs = spacepos + 1;\n }\n subprocess->addArgument(libs);\n }\n\n auto logStream = subprocess->captureStdoutAndStderr();\n\n auto subprocessWaitOp = eventManager->when(subprocess->start(eventManager))(\n [context](ProcessExitCode exitCode) {\n if (exitCode.wasSignaled() || exitCode.getExitCode() != 0) {\n context->failed();\n }\n });\n\n auto logger = newOwned(context, logStream.release());\n auto logOp = logger->run(eventManager);\n\n return eventManager->when(subprocessWaitOp, logOp, logger, subprocess, executableFile)(\n [](Void, Void, OwnedPtr, OwnedPtr, OwnedPtr){});\n}\n\n// =======================================================================================\n\nconst Tag CppActionFactory::MAIN_SYMBOLS[] = {\n Tag::fromName(\"c++symbol:main\"),\n Tag::fromName(\"c++symbol:_main\")\n};\n\nconst Tag CppActionFactory::GTEST_TEST = Tag::fromName(\"gtest:test\");\nconst Tag CppActionFactory::KJTEST_TEST = Tag::fromName(\"kjtest:test\");\nconst Tag CppActionFactory::NODEJS_MODULE = Tag::fromName(\"nodejs:module\");\nconst Tag CppActionFactory::LIBFUZZER_SYMBOL = Tag::fromName(\"c++symbol:LLVMFuzzerTestOneInput\");\n\nCppActionFactory::CppActionFactory() {}\nCppActionFactory::~CppActionFactory() {}\n\nvoid CppActionFactory::enumerateTriggerTags(\n std::back_insert_iterator > iter) {\n for (unsigned int i = 0; i < (sizeof(MAIN_SYMBOLS) / sizeof(MAIN_SYMBOLS[0])); i++) {\n *iter++ = MAIN_SYMBOLS[i];\n }\n *iter++ = GTEST_TEST;\n *iter++ = KJTEST_TEST;\n *iter++ = NODEJS_MODULE;\n if (getenv(\"EKAM_LIBFUZZER_ENABLE\") != nullptr) {\n *iter++ = LIBFUZZER_SYMBOL;\n }\n}\n\nOwnedPtr CppActionFactory::tryMakeAction(const Tag& id, File* file) {\n OwnedPtr result;\n for (unsigned int i = 0; i < (sizeof(MAIN_SYMBOLS) / sizeof(MAIN_SYMBOLS[0])); i++) {\n if (id == MAIN_SYMBOLS[i]) {\n return newOwned(file, LinkAction::NORMAL);\n }\n }\n if (id == LIBFUZZER_SYMBOL) {\n return newOwned(file, LinkAction::LIBFUZZER);\n }\n if (id == GTEST_TEST) {\n return newOwned(file, LinkAction::GTEST);\n }\n if (id == KJTEST_TEST) {\n return newOwned(file, LinkAction::KJTEST);\n }\n if (id == NODEJS_MODULE) {\n return newOwned(file, LinkAction::NODEJS);\n }\n return nullptr;\n}\n\n} // namespace ekam\n" }, { "path": "src/ekam/CppActionFactory.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_EKAM_CPPACTIONFACTORY_H_\n#define KENTONSCODE_EKAM_CPPACTIONFACTORY_H_\n\n#include \n#include \n#include \"Action.h\"\n\nnamespace ekam {\n\nclass CppActionFactory: public ActionFactory {\npublic:\n CppActionFactory();\n ~CppActionFactory();\n\n // implements ActionFactory ------------------------------------------------------------\n void enumerateTriggerTags(std::back_insert_iterator > iter);\n OwnedPtr tryMakeAction(const Tag& id, File* file);\n\nprivate:\n static const Tag MAIN_SYMBOLS[];\n static const Tag LIBFUZZER_SYMBOL;\n static const Tag GTEST_TEST;\n static const Tag KJTEST_TEST;\n static const Tag NODEJS_MODULE;\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_EKAM_CPPACTIONFACTORY_H_\n" }, { "path": "src/ekam/Dashboard.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"Dashboard.h\"\n\nnamespace ekam {\n\nDashboard::~Dashboard() {}\nDashboard::Task::~Task() {}\n\n}\n" }, { "path": "src/ekam/Dashboard.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_EKAM_DASHBOARD_H_\n#define KENTONSCODE_EKAM_DASHBOARD_H_\n\n#include \n#include \"base/OwnedPtr.h\"\n\nnamespace ekam {\n\nclass Dashboard {\npublic:\n virtual ~Dashboard();\n\n enum TaskState {\n PENDING, // Default state.\n RUNNING,\n DONE,\n PASSED, // Like DONE, but should be displayed prominently (hint: test result).\n FAILED,\n BLOCKED\n };\n\n class Task {\n public:\n virtual ~Task();\n\n virtual void setState(TaskState state) = 0;\n virtual void addOutput(const std::string& text) = 0;\n };\n\n enum Silence {\n SILENT,\n NORMAL\n };\n\n virtual OwnedPtr beginTask(const std::string& verb, const std::string& noun,\n Silence silence) = 0;\n};\n\nclass EventManager;\n\nOwnedPtr initNetworkDashboard(EventManager* eventManager, const std::string& address,\n OwnedPtr dashboardToWrap);\n\n} // namespace ekam\n\n#endif // KENTONSCODE_EKAM_DASHBOARD_H_\n" }, { "path": "src/ekam/Driver.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"Driver.h\"\n\n#include \n#include \n#include \n#include \n#include \n#include \n\n#include \"base/Debug.h\"\n#include \"os/EventGroup.h\"\n\nnamespace ekam {\n\nnamespace {\n\nint fileDepth(const std::string& name) {\n int result = 0;\n for (unsigned int i = 0; i < name.size(); i++) {\n if (name[i] == '/') {\n ++result;\n }\n }\n return result;\n}\n\nint commonPrefixLength(const std::string& srcName, const std::string& bestMatchName) {\n std::string::size_type n = std::min(srcName.size(), bestMatchName.size());\n for (unsigned int i = 0; i < n; i++) {\n if (srcName[i] != bestMatchName[i]) {\n return i;\n }\n }\n return n;\n}\n\n} // namespace\n\nclass Driver::ActionDriver : public BuildContext, public EventGroup::ExceptionHandler {\npublic:\n ActionDriver(Driver* driver, OwnedPtr action,\n File* srcfile, Hash srcHash, OwnedPtr task);\n ~ActionDriver();\n\n void start();\n\n // implements BuildContext -------------------------------------------------------------\n File* findProvider(Tag id);\n File* findInput(const std::string& path);\n\n void provide(File* file, const std::vector& tags);\n void install(File* file, InstallLocation location, const std::string& name);\n void log(const std::string& text);\n\n OwnedPtr newOutput(const std::string& path);\n\n void addActionType(OwnedPtr factory);\n\n void passed();\n void failed();\n\n // implements ExceptionHandler ---------------------------------------------------------\n void threwException(const std::exception& e);\n void threwUnknownException();\n void noMoreEvents();\n\nprivate:\n Driver* driver;\n OwnedPtr action;\n OwnedPtr srcfile;\n Hash srcHash;\n OwnedPtr dashboardTask;\n\n // TODO: Get rid of \"state\". Maybe replace with \"status\" or something, but don't try to\n // track both whether we're running and what the status was at the same time. (I already\n // had to split isRunning into a separate boolean due to issues with this.)\n enum {\n PENDING,\n RUNNING,\n DONE,\n PASSED,\n FAILED\n } state;\n\n EventGroup eventGroup;\n\n Promise asyncCallbackOp;\n\n bool isRunning;\n Promise runningAction;\n\n OwnedPtrVector outputs;\n\n struct Installation {\n File* file;\n InstallLocation location;\n std::string name;\n };\n std::vector installations;\n\n OwnedPtrVector provisions;\n OwnedPtrVector > providedTags;\n OwnedPtrVector providedFactories;\n\n // True if returned() is currently on the stack. Causes destructor to abort. Used for\n // debugging.\n bool currentlyExecutingReturned = false;\n\n void ensureRunning();\n void queueDoneCallback();\n void returned();\n void reset();\n Provision* choosePreferredProvider(const Tag& tag);\n File* provideInternal(File* file, const std::vector& tags);\n\n friend class Driver;\n};\n\nDriver::ActionDriver::ActionDriver(Driver* driver, OwnedPtr action,\n File* srcfile, Hash srcHash,\n OwnedPtr task)\n : driver(driver), action(action.release()), srcfile(srcfile->clone()), srcHash(srcHash),\n dashboardTask(task.release()), state(PENDING), eventGroup(driver->eventManager, this),\n isRunning(false) {}\nDriver::ActionDriver::~ActionDriver() {\n assert(!currentlyExecutingReturned);\n}\n\nvoid Driver::ActionDriver::start() {\n assert(state == PENDING);\n assert(!driver->dependencyTable.has(this));\n assert(outputs.empty());\n assert(provisions.empty());\n assert(installations.empty());\n assert(providedFactories.empty());\n assert(!isRunning);\n\n state = RUNNING;\n isRunning = true;\n dashboardTask->setState(Dashboard::RUNNING);\n\n asyncCallbackOp = eventGroup.when()(\n [this]() {\n asyncCallbackOp.release();\n runningAction = action->start(&eventGroup, this);\n });\n}\n\nFile* Driver::ActionDriver::findProvider(Tag tag) {\n ensureRunning();\n\n Provision* provision = choosePreferredProvider(tag);\n\n driver->dependencyTable.add(tag, this, provision);\n if (provision == NULL) {\n return NULL;\n } else {\n return provision->file.get();\n }\n}\n\nFile* Driver::ActionDriver::findInput(const std::string& path) {\n ensureRunning();\n\n return findProvider(Tag::fromFile(path));\n}\n\nvoid Driver::ActionDriver::provide(File* file, const std::vector& tags) {\n provideInternal(file, tags);\n}\n\nFile* Driver::ActionDriver::provideInternal(File* file, const std::vector& tags) {\n ensureRunning();\n\n // Find existing provision for this file, if any.\n // TODO: Convert provisions into a map?\n Provision* provision = NULL;\n for (int i = 0; i < provisions.size(); i++) {\n if (provisions.get(i)->file->equals(file)) {\n provision = provisions.get(i);\n providedTags.get(i)->insert(providedTags.get(i)->end(), tags.begin(), tags.end());\n break;\n }\n }\n\n if (provision == NULL) {\n auto ownedProvision = newOwned();\n provision = ownedProvision.get();\n provision->creator = this;\n provisions.add(ownedProvision.release());\n providedTags.add(newOwned>(tags));\n }\n\n provision->file = file->clone();\n return provision->file.get();\n}\n\nvoid Driver::ActionDriver::install(File* file, InstallLocation location, const std::string& name) {\n ensureRunning();\n\n std::string tagName(INSTALL_LOCATION_NAMES[location]);\n tagName.push_back(':');\n tagName.append(name);\n std::vector tags;\n tags.push_back(Tag::fromName(tagName));\n File* ownedFile = provideInternal(file, tags);\n\n Installation installation = { ownedFile, location, name };\n installations.push_back(installation);\n}\n\nvoid Driver::ActionDriver::log(const std::string& text) {\n ensureRunning();\n dashboardTask->addOutput(text);\n}\n\nOwnedPtr Driver::ActionDriver::newOutput(const std::string& path) {\n ensureRunning();\n OwnedPtr file = driver->tmp->relative(path);\n\n recursivelyCreateDirectory(file->parent().get());\n\n OwnedPtr result = file->clone();\n\n std::vector tags;\n tags.push_back(Tag::DEFAULT_TAG);\n provide(file.get(), tags);\n\n outputs.add(file.release());\n\n return result;\n}\n\nvoid Driver::ActionDriver::addActionType(OwnedPtr factory) {\n ensureRunning();\n providedFactories.add(factory.release());\n}\n\nvoid Driver::ActionDriver::noMoreEvents() {\n if (isRunning) {\n if (state == RUNNING) {\n state = DONE;\n queueDoneCallback();\n }\n }\n}\n\nvoid Driver::ActionDriver::passed() {\n ensureRunning();\n\n if (state == FAILED) {\n // Ignore passed() after failed().\n return;\n }\n\n state = PASSED;\n queueDoneCallback();\n}\n\nvoid Driver::ActionDriver::failed() {\n ensureRunning();\n\n if (state == FAILED) {\n // Ignore redundant call to failed().\n return;\n } else if (state == DONE) {\n // (done callback should already be queued)\n throw std::runtime_error(\"Called failed() after success().\");\n } else if (state == PASSED) {\n // (done callback should already be queued)\n throw std::runtime_error(\"Called failed() after passed().\");\n } else {\n state = FAILED;\n queueDoneCallback();\n }\n}\n\nvoid Driver::ActionDriver::ensureRunning() {\n if (!isRunning) {\n throw std::runtime_error(\"Action is not running.\");\n }\n}\n\nvoid Driver::ActionDriver::queueDoneCallback() {\n asyncCallbackOp = driver->eventManager->when()(\n [this]() {\n asyncCallbackOp.release();\n Driver* driver = this->driver;\n returned(); // may delete this\n driver->startSomeActions();\n });\n}\n\nvoid Driver::ActionDriver::threwException(const std::exception& e) {\n ensureRunning();\n dashboardTask->addOutput(std::string(\"uncaught exception: \") + e.what() + \"\\n\");\n asyncCallbackOp.release();\n state = FAILED;\n returned();\n}\n\nvoid Driver::ActionDriver::threwUnknownException() {\n ensureRunning();\n dashboardTask->addOutput(\"uncaught exception of unknown type\\n\");\n asyncCallbackOp.release();\n state = FAILED;\n returned();\n}\n\n// Poor man's kj::defer.\ntemplate\nclass Deferred {\npublic:\n inline Deferred(Func&& func): func(func), canceled(false) {}\n inline ~Deferred() noexcept(false) { if (!canceled) func(); }\n\n // This move constructor is usually optimized away by the compiler.\n inline Deferred(Deferred&& other): func(other.func), canceled(false) {\n other.canceled = true;\n }\nprivate:\n Func func;\n bool canceled;\n};\n\ntemplate \nDeferred defer(Func&& func) {\n // Returns an object which will invoke the given functor in its destructor. The object is not\n // copyable but is movable with the semantics you'd expect. Since the return type is private,\n // you need to assign to an `auto` variable.\n return Deferred(std::forward(func));\n}\n\n\nvoid Driver::ActionDriver::returned() {\n ensureRunning();\n\n currentlyExecutingReturned = true;\n auto _ = defer([this]() {\n currentlyExecutingReturned = false;\n });\n\n // Cancel anything still running.\n runningAction.release();\n isRunning = false;\n\n // Pull self out of driver->activeActions.\n OwnedPtr self;\n for (int i = 0; i < driver->activeActions.size(); i++) {\n if (driver->activeActions.get(i) == this) {\n self = driver->activeActions.releaseAndShift(i);\n break;\n }\n }\n\n driver->completedActionPtrs.add(this, self.release());\n\n if (state == FAILED) {\n // Failed, possibly due to missing dependencies.\n provisions.clear();\n installations.clear();\n providedTags.clear();\n providedFactories.clear();\n outputs.clear();\n dashboardTask->setState(Dashboard::BLOCKED);\n } else {\n dashboardTask->setState(state == PASSED ? Dashboard::PASSED : Dashboard::DONE);\n\n // Remove outputs which were deleted before the action completed. Some actions create\n // files and then delete them immediately.\n OwnedPtrVector provisionsToFilter;\n provisions.swap(&provisionsToFilter);\n for (int i = 0; i < provisionsToFilter.size(); i++) {\n if (provisionsToFilter.get(i)->file->exists()) {\n provisions.add(provisionsToFilter.release(i));\n }\n }\n\n // Register providers. But, don't allow our own dependencies to depend on them.\n std::unordered_set deps;\n driver->getTransitiveDependencies(this, &deps);\n for (int i = 0; i < provisions.size(); i++) {\n driver->registerProvider(provisions.get(i), *providedTags.get(i), deps);\n }\n providedTags.clear(); // Not needed anymore.\n\n // Register factories.\n for (int i = 0; i < providedFactories.size(); i++) {\n driver->addActionFactory(providedFactories.get(i));\n driver->rescanForNewFactory(providedFactories.get(i));\n }\n\n // Install files.\n for (size_t i = 0; i < installations.size(); i++) {\n File* installDir = driver->installDirs[installations[i].location];\n OwnedPtr target = installDir->relative(installations[i].name);\n if (target->exists()) {\n target->unlink();\n } else {\n if (!installDir->isDirectory()) {\n // Can't rely on recursivelyCreateDirectory() for the root directory because it will\n // call parent() on it which will throw.\n installDir->createDirectory();\n }\n recursivelyCreateDirectory(target->parent().get());\n }\n target->link(installations[i].file);\n }\n }\n}\n\nvoid Driver::ActionDriver::reset() {\n assert(!currentlyExecutingReturned);\n\n if (state == PENDING) {\n // Nothing to do.\n return;\n }\n\n OwnedPtr self;\n\n if (isRunning) {\n dashboardTask->setState(Dashboard::BLOCKED);\n runningAction.release();\n asyncCallbackOp.release();\n\n for (int i = 0; i < driver->activeActions.size(); i++) {\n if (driver->activeActions.get(i) == this) {\n self = driver->activeActions.releaseAndShift(i);\n break;\n }\n }\n\n isRunning = false;\n } else {\n if (!driver->completedActionPtrs.release(this, &self)) {\n throw std::logic_error(\"Action not running or pending, but not in completedActionPtrs?\");\n }\n }\n\n state = PENDING;\n\n // Put on back of queue (as opposed to front) so that actions which are frequently reset\n // don't get redundantly rebuilt too much. We add the action to the queue before resetting\n // dependents so that this action gets re-run before its dependents.\n // TODO: The second point probably doesn't help much when multiprocessing. Maybe the\n // action queue should really be a graph that remembers what depended on what the last\n // time we ran them, and avoids re-running any action before re-running actions on which it\n // depended last time.\n driver->pendingActions.pushBack(self.release());\n\n // Reset dependents.\n for (int i = 0; i < provisions.size(); i++) {\n driver->resetDependentActions(provisions.get(i));\n }\n\n // Actions created by any provided ActionFactories must be deleted.\n for (int i = 0; i < providedFactories.size(); i++) {\n ActionFactory* factory = providedFactories.get(i);\n\n std::vector actionsToDelete;\n for (ActionTriggersTable::SearchIterator\n iter(driver->actionTriggersTable, factory); iter.next();) {\n // Can't call reset() directly here because it may invalidate our iterator.\n actionsToDelete.push_back(iter.cell());\n }\n\n for (size_t j = 0; j < actionsToDelete.size(); j++) {\n actionsToDelete[j]->reset();\n\n // TODO: Use better data structure for pendingActions. For now we have to iterate\n // through the whole thing to find the action we're deleting. We iterate from the back\n // since it's likely the action was just added there.\n for (int k = driver->pendingActions.size() - 1; k >= 0; k--) {\n if (driver->pendingActions.get(k) == actionsToDelete[j]) {\n driver->pendingActions.releaseAndShift(k);\n break;\n }\n }\n }\n\n driver->actionTriggersTable.erase(factory);\n driver->triggers.erase(factory);\n }\n\n // Remove all entries in dependencyTable pointing at this action.\n driver->dependencyTable.erase(this);\n\n provisions.clear();\n installations.clear();\n providedTags.clear();\n providedFactories.clear();\n outputs.clear();\n}\n\nDriver::Provision* Driver::ActionDriver::choosePreferredProvider(const Tag& tag) {\n TagTable::SearchIterator iter(driver->tagTable, tag);\n\n if (!iter.next()) {\n return NULL;\n } else {\n std::string srcName = srcfile->canonicalName();\n Provision* bestMatch = iter.cell();\n\n if (iter.next()) {\n // There are multiple files with this tag. We must choose which one we like best.\n std::string bestMatchName = bestMatch->file->canonicalName();\n int bestMatchDepth = fileDepth(bestMatchName);\n int bestMatchCommonPrefix = commonPrefixLength(srcName, bestMatchName);\n\n do {\n Provision* candidate = iter.cell();\n std::string candidateName = candidate->file->canonicalName();\n int candidateDepth = fileDepth(candidateName);\n int candidateCommonPrefix = commonPrefixLength(srcName, candidateName);\n if (candidateCommonPrefix < bestMatchCommonPrefix) {\n // Prefer provider that is closer in the directory tree.\n continue;\n } else if (candidateCommonPrefix == bestMatchCommonPrefix) {\n if (candidateDepth > bestMatchDepth) {\n // Prefer provider that is less deeply nested.\n continue;\n } else if (candidateDepth == bestMatchDepth) {\n // Arbitrarily -- but consistently -- choose one.\n int diff = bestMatchName.compare(candidateName);\n if (diff < 0) {\n // Prefer file that comes first alphabetically.\n continue;\n } else if (diff == 0) {\n // TODO: Is this really an error? I think it is for the moment, but someday it\n // may not be, if multiple actions are allowed to produce outputs with the same\n // canonical names.\n DEBUG_ERROR << \"Two providers have same file name: \" << bestMatchName;\n continue;\n }\n }\n }\n\n // If we get here, the candidate is better than the existing best match.\n bestMatch = candidate;\n bestMatchName.swap(candidateName);\n bestMatchDepth = candidateDepth;\n bestMatchCommonPrefix = candidateCommonPrefix;\n } while(iter.next());\n }\n\n return bestMatch;\n }\n}\n\n// =======================================================================================\n\nDriver::Driver(EventManager* eventManager, Dashboard* dashboard, File* tmp,\n File* installDirs[BuildContext::INSTALL_LOCATION_COUNT], int maxConcurrentActions,\n ActivityObserver* activityObserver)\n : eventManager(eventManager), dashboard(dashboard), tmp(tmp),\n maxConcurrentActions(maxConcurrentActions), activityObserver(activityObserver) {\n if (!tmp->isDirectory()) {\n tmp->createDirectory();\n }\n\n for (int i = 0; i < BuildContext::INSTALL_LOCATION_COUNT; i++) {\n this->installDirs[i] = installDirs[i];\n }\n}\n\nDriver::~Driver() {}\n\nvoid Driver::addActionFactory(ActionFactory* factory) {\n std::vector triggerTags;\n factory->enumerateTriggerTags(std::back_inserter(triggerTags));\n for (unsigned int i = 0; i < triggerTags.size(); i++) {\n triggers.add(triggerTags[i], factory);\n }\n}\n\nvoid Driver::addSourceFile(File* file) {\n OwnedPtr provision;\n if (rootProvisions.release(file, &provision)) {\n // Source file was modified. Reset all actions dependent on the old version.\n resetDependentActions(provision.get());\n }\n\n // Apply default tag.\n std::vector tags;\n tags.push_back(Tag::DEFAULT_TAG);\n\n provision = newOwned();\n provision->creator = nullptr;\n provision->file = file->clone();\n registerProvider(provision.get(), tags, std::unordered_set());\n File* key = provision->file.get(); // cannot inline due to undefined evaluation order\n rootProvisions.add(key, provision.release());\n\n startSomeActions();\n}\n\nvoid Driver::removeSourceFile(File* file) {\n OwnedPtr provision;\n if (rootProvisions.release(file, &provision)) {\n resetDependentActions(provision.get());\n\n // In case some active actions were canceled.\n startSomeActions();\n } else {\n DEBUG_ERROR << \"Tried to remove source file that wasn't ever added: \" << file->canonicalName();\n }\n}\n\nvoid Driver::startSomeActions() {\n while (activeActions.size() < maxConcurrentActions && !pendingActions.empty()) {\n if (activityObserver != nullptr) activityObserver->startingAction();\n OwnedPtr actionDriver = pendingActions.popFront();\n ActionDriver* ptr = actionDriver.get();\n activeActions.add(actionDriver.release());\n try {\n ptr->start();\n } catch (const std::exception& e) {\n ptr->threwException(e);\n } catch (...) {\n ptr->threwUnknownException();\n }\n }\n\n if (activeActions.size() == 0) {\n bool hasFailures = dumpErrors();\n if (activityObserver != nullptr) activityObserver->idle(hasFailures);\n }\n}\n\nvoid Driver::rescanForNewFactory(ActionFactory* factory) {\n // Apply triggers.\n std::vector triggerTags;\n factory->enumerateTriggerTags(std::back_inserter(triggerTags));\n for (unsigned int i = 0; i < triggerTags.size(); i++) {\n for (TagTable::SearchIterator iter(tagTable, triggerTags[i]); iter.next();) {\n Provision* provision = iter.cell();\n OwnedPtr action = factory->tryMakeAction(triggerTags[i], provision->file.get());\n if (action != NULL) {\n queueNewAction(factory, action.release(), provision);\n }\n }\n }\n}\n\nvoid Driver::queueNewAction(ActionFactory* factory, OwnedPtr action,\n Provision* provision) {\n OwnedPtr task = dashboard->beginTask(\n action->getVerb(), provision->file->canonicalName(),\n action->isSilent() ? Dashboard::SILENT : Dashboard::NORMAL);\n\n OwnedPtr actionDriver =\n newOwned(this, action.release(), provision->file.get(), provision->contentHash,\n task.release());\n actionTriggersTable.add(factory, provision, actionDriver.get());\n\n // Put new action on front of queue because it was probably triggered by another action that\n // just completed, and it's good to run related actions together to improve cache locality.\n pendingActions.pushFront(actionDriver.release());\n}\n\nvoid Driver::getTransitiveDependencies(\n ActionDriver* action, std::unordered_set* deps) {\n if (action != nullptr && deps->insert(action).second) {\n for (ActionTriggersTable::SearchIterator\n iter(actionTriggersTable, action); iter.next();) {\n getTransitiveDependencies(iter.cell()->creator, deps);\n }\n for (DependencyTable::SearchIterator\n iter(dependencyTable, action); iter.next();) {\n Provision* provision = iter.cell();\n if (provision != nullptr) {\n getTransitiveDependencies(provision->creator, deps);\n }\n }\n }\n}\n\nvoid Driver::registerProvider(Provision* provision, const std::vector& tags,\n const std::unordered_set& dependencies) {\n provision->contentHash = provision->file->contentHash();\n\n for (std::vector::const_iterator iter = tags.begin(); iter != tags.end(); ++iter) {\n const Tag& tag = *iter;\n tagTable.add(tag, provision);\n\n resetDependentActions(tag, dependencies);\n\n fireTriggers(tag, provision);\n }\n}\n\nvoid Driver::resetDependentActions(const Tag& tag,\n const std::unordered_set& dependencies) {\n std::unordered_set provisionsToReset;\n\n std::vector actionsToReset;\n\n for (DependencyTable::SearchIterator iter(dependencyTable, tag);\n iter.next();) {\n ActionDriver* action = iter.cell();\n\n // Don't reset an action that contributed to the creation of this tag in the first place, since\n // that would lead to an infinite loop of rebuilding the same action.\n if (dependencies.count(action) == 0) {\n Provision* previousProvider = iter.cell();\n\n if (action->choosePreferredProvider(tag) != previousProvider) {\n // We can't just call reset() here because it could invalidate our iterator.\n actionsToReset.push_back(action);\n }\n } else {\n DEBUG_INFO << \"Action's inputs are affected by its outputs.\";\n }\n }\n\n for (size_t i = 0; i < actionsToReset.size(); i++) {\n // Only reset the action if it is still in the dependency table. If not, it was already\n // reset (and possibly deleted!) elsewhere.\n if (dependencyTable.find(actionsToReset[i]) != nullptr) {\n actionsToReset[i]->reset();\n }\n }\n}\n\nvoid Driver::resetDependentActions(Provision* provision) {\n // Reset dependents of this provision.\n {\n std::vector actionsToReset;\n for (DependencyTable::SearchIterator\n iter(dependencyTable, provision); iter.next();) {\n // Can't call reset() directly here because it may invalidate our iterator.\n actionsToReset.push_back(iter.cell());\n }\n for (size_t j = 0; j < actionsToReset.size(); j++) {\n // Only reset the action if it is still in the dependency table. If not, it was already\n // reset (and possibly deleted!) elsewhere.\n if (dependencyTable.find(actionsToReset[j]) != nullptr) {\n actionsToReset[j]->reset();\n }\n }\n if (dependencyTable.erase(provision) > 0) {\n DEBUG_ERROR << \"Resetting dependents should have removed this provision from \"\n \"dependencyTable.\";\n }\n }\n\n // Everything triggered by this provision must be deleted.\n {\n std::vector actionsToDelete;\n\n for (ActionTriggersTable::SearchIterator\n iter(actionTriggersTable, provision); iter.next();) {\n // Can't call reset() directly here because it may invalidate our iterator.\n actionsToDelete.push_back(iter.cell());\n }\n\n for (size_t j = 0; j < actionsToDelete.size(); j++) {\n actionsToDelete[j]->reset();\n\n // TODO: Use better data structure for pendingActions. For now we have to iterate\n // through the whole thing to find the action we're deleting. We iterate from the back\n // since it's likely the action was just added there.\n for (int k = pendingActions.size() - 1; k >= 0; k--) {\n if (pendingActions.get(k) == actionsToDelete[j]) {\n pendingActions.releaseAndShift(k);\n break;\n }\n }\n }\n\n actionTriggersTable.erase(provision);\n }\n\n tagTable.erase(provision);\n}\n\nvoid Driver::fireTriggers(const Tag& tag, Provision* provision) {\n for (TriggerTable::SearchIterator iter(triggers, tag); iter.next();) {\n ActionFactory* factory = iter.cell();\n OwnedPtr triggeredAction = factory->tryMakeAction(tag, provision->file.get());\n if (triggeredAction != NULL) {\n queueNewAction(factory, triggeredAction.release(), provision);\n }\n }\n}\n\nbool Driver::dumpErrors() {\n bool hasFailures = false;\n for (OwnedPtrMap::Iterator iter(completedActionPtrs); iter.next();) {\n if (iter.key()->state == ActionDriver::FAILED) {\n iter.value()->dashboardTask->setState(Dashboard::FAILED);\n hasFailures = true;\n }\n }\n return hasFailures;\n}\n\n} // namespace ekam\n" }, { "path": "src/ekam/Driver.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_EKAM_DRIVER_H_\n#define KENTONSCODE_EKAM_DRIVER_H_\n\n#include \n#include \n#include \n#include \n\n#include \"base/OwnedPtr.h\"\n#include \"os/File.h\"\n#include \"Action.h\"\n#include \"Tag.h\"\n#include \"Dashboard.h\"\n#include \"base/Table.h\"\n\nnamespace ekam {\n\nclass Driver {\npublic:\n class ActivityObserver {\n public:\n virtual void startingAction() = 0;\n virtual void idle(bool hasFailures) = 0;\n };\n\n Driver(EventManager* eventManager, Dashboard* dashboard, File* tmp,\n File* installDirs[BuildContext::INSTALL_LOCATION_COUNT], int maxConcurrentActions,\n ActivityObserver* activityObserver = nullptr);\n ~Driver();\n\n void addActionFactory(ActionFactory* factory);\n\n void addSourceFile(File* file);\n void removeSourceFile(File* file);\n\nprivate:\n class ActionDriver;\n\n EventManager* eventManager;\n Dashboard* dashboard;\n\n File* tmp;\n File* installDirs[BuildContext::INSTALL_LOCATION_COUNT];\n\n int maxConcurrentActions;\n\n ActivityObserver* activityObserver;\n\n class TriggerTable : public Table,\n IndexedColumn > {\n public:\n static const int TAG = 0;\n static const int FACTORY = 1;\n };\n TriggerTable triggers;\n\n struct Provision {\n ActionDriver* creator; // possibly null\n OwnedPtr file;\n Hash contentHash;\n };\n\n class TagTable : public Table, IndexedColumn > {\n public:\n static const int TAG = 0;\n static const int PROVISION = 1;\n };\n TagTable tagTable;\n\n OwnedPtrVector activeActions;\n OwnedPtrDeque pendingActions;\n OwnedPtrMap completedActionPtrs;\n\n class DependencyTable : public Table,\n IndexedColumn,\n IndexedColumn > {\n public:\n static const int TAG = 0;\n static const int ACTION = 1;\n static const int PROVISION = 2;\n };\n DependencyTable dependencyTable;\n\n class ActionTriggersTable : public Table,\n IndexedColumn,\n IndexedColumn > {\n public:\n static const int FACTORY = 0;\n static const int PROVISION = 1;\n static const int ACTION = 2;\n };\n ActionTriggersTable actionTriggersTable;\n\n OwnedPtrMap rootProvisions;\n\n void startSomeActions();\n\n void rescanForNewFactory(ActionFactory* factory);\n\n void queueNewAction(ActionFactory* factory, OwnedPtr action,\n Provision* provision);\n\n void getTransitiveDependencies(ActionDriver* action, std::unordered_set* deps);\n\n void registerProvider(Provision* provision, const std::vector& tags,\n const std::unordered_set& dependencies);\n void resetDependentActions(const Tag& tag,\n const std::unordered_set& dependencies);\n void resetDependentActions(Provision* provision);\n void fireTriggers(const Tag& tag, Provision* provision);\n\n bool dumpErrors();\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_EKAM_DRIVER_H_\n" }, { "path": "src/ekam/ExecPluginActionFactory.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"ExecPluginActionFactory.h\"\n\n#include \n#include \n#include \n\n#include \"os/Subprocess.h\"\n#include \"ActionUtil.h\"\n#include \"base/Debug.h\"\n\nnamespace ekam {\n\nnamespace {\n\nstd::string splitToken(std::string* line) {\n std::string::size_type pos = line->find_first_of(' ');\n std::string result;\n if (pos == std::string::npos) {\n result = *line;\n line->clear();\n } else {\n result.assign(*line, 0, pos);\n line->erase(0, pos + 1);\n }\n return result;\n}\n\n} // namespace\n\n// =======================================================================================\n\nclass PluginDerivedActionFactory : public ActionFactory {\npublic:\n PluginDerivedActionFactory(OwnedPtr executable,\n std::string&& verb,\n bool silent,\n std::vector&& triggers);\n ~PluginDerivedActionFactory();\n\n // implements ActionFactory -----------------------------------------------------------\n void enumerateTriggerTags(std::back_insert_iterator > iter);\n OwnedPtr tryMakeAction(const Tag& id, File* file);\n\nprivate:\n OwnedPtr executable;\n std::string verb;\n bool silent;\n std::vector triggers;\n};\n\n// =======================================================================================\n\nclass PluginDerivedAction : public Action {\npublic:\n PluginDerivedAction(File* executable, const std::string& verb, bool silent, File* file)\n : executable(executable->clone()), verb(verb), silent(silent) {\n if (file != NULL) {\n this->file = file->clone();\n }\n }\n ~PluginDerivedAction() {}\n\n // implements Action -------------------------------------------------------------------\n std::string getVerb() { return verb; }\n bool isSilent() { return silent; }\n Promise start(EventManager* eventManager, BuildContext* context);\n\nprivate:\n class CommandReader;\n\n OwnedPtr executable;\n std::string verb;\n bool silent;\n OwnedPtr file; // nullable\n};\n\nclass PluginDerivedAction::CommandReader {\npublic:\n CommandReader(BuildContext* context, OwnedPtr requestStream,\n OwnedPtr responseStream, File* executable, File* input)\n : context(context), executable(executable->clone()),\n requestStream(requestStream.release()),\n responseStream(responseStream.release()),\n lineReader(this->requestStream.get()), silent(false) {\n if (input != NULL) {\n this->input = input->clone();\n knownFiles.add(input->canonicalName(), input->clone());\n }\n\n std::string junk;\n splitExtension(executable->basename(), &verb, &junk);\n }\n ~CommandReader() {}\n\n Promise readAll(EventManager* eventManager) {\n return eventManager->when(lineReader.readLine(eventManager))(\n [=](OwnedPtr line) -> Promise {\n if (line == nullptr) {\n eof();\n return newFulfilledPromise();\n }\n\n consume(*line);\n return readAll(eventManager);\n }, [=](MaybeException> error) {\n try {\n error.get();\n } catch (const std::exception& e) {\n context->log(e.what());\n context->failed();\n throw;\n } catch (...) {\n context->log(\"unknown exception\");\n context->failed();\n throw;\n }\n });\n }\n\nprivate:\n void consume(const std::string& line) {\n if (findInCache(line)) return;\n\n std::string args = line;\n std::string command = splitToken(&args);\n\n if (command == \"verb\") {\n verb = args;\n } else if (command == \"silent\") {\n silent = true;\n } else if (command == \"trigger\") {\n triggers.push_back(Tag::fromName(args));\n } else if (command == \"findProvider\" || command == \"findInput\") {\n File* provider;\n if (command == \"findProvider\") {\n provider = context->findProvider(Tag::fromName(args));\n } else if (input != NULL && args == input->canonicalName()) {\n provider = input.get();\n } else if (findInCache(\"newOutput \" + args)) {\n // File was originally created by this action. findInCache() already wrote the path,\n // so just return.\n return;\n } else {\n provider = context->findInput(args);\n }\n if (provider != NULL) {\n OwnedPtr diskRef = provider->getOnDisk(File::READ);\n std::string path = diskRef->path();\n cache.insert(std::make_pair(line, diskRef.get()));\n diskRefs.add(diskRef.release());\n responseStream->writeAll(path.data(), path.size());\n\n knownFiles.add(path, provider->clone());\n }\n responseStream->writeAll(\"\\n\", 1);\n } else if (command == \"findModifiers\") {\n auto dir = input->parent();\n std::vector results;\n for (;;) {\n File* provider = context->findProvider(Tag::fromName(\n \"canonical:\" + dir->relative(args)->canonicalName()));\n if (provider != NULL) {\n results.push_back(provider);\n }\n if (!dir->hasParent()) {\n break;\n }\n dir = dir->parent();\n }\n\n for (auto iter = results.rbegin(); iter != results.rend(); ++iter) {\n File* provider = *iter;\n OwnedPtr diskRef = provider->getOnDisk(File::READ);\n std::string path = diskRef->path();\n diskRefs.add(diskRef.release());\n responseStream->writeAll(path.data(), path.size());\n knownFiles.add(path, provider->clone());\n responseStream->writeAll(\"\\n\", 1);\n }\n\n responseStream->writeAll(\"\\n\", 1);\n } else if (command == \"newProvider\") {\n // TODO: Create a new output file and register it as a provider.\n context->log(\"newProvider not implemented\");\n context->failed();\n } else if (command == \"noteInput\") {\n // The action is reading some file outside the working directory. For now we ignore this.\n // TODO: Pay attention? We could trigger rebuilds when installed tools are updated, etc.\n } else if (command == \"newOutput\") {\n OwnedPtr file = context->newOutput(args);\n\n OwnedPtr diskRef = file->getOnDisk(File::WRITE);\n std::string path = diskRef->path();\n\n cache.insert(std::make_pair(line, diskRef.get()));\n diskRefs.add(diskRef.release());\n knownFiles.add(path, file.release());\n\n responseStream->writeAll(path.data(), path.size());\n responseStream->writeAll(\"\\n\", 1);\n } else if (command == \"provide\") {\n std::string filename = splitToken(&args);\n File* file = knownFiles.get(filename);\n if (file == NULL) {\n context->log(\"File passed to \\\"provide\\\" not created with \\\"newOutput\\\" nor noted as an \"\n \"input: \" + filename + \"\\n\");\n context->failed();\n } else {\n provisions.insert(std::make_pair(file, Tag::fromName(args)));\n }\n } else if (command == \"install\") {\n std::string filename = splitToken(&args);\n File* file = knownFiles.get(filename);\n\n if (file == NULL) {\n context->log(\"File passed to \\\"install\\\" not created with \\\"newOutput\\\" nor noted as an \"\n \"input: \" + filename + \"\\n\");\n context->failed();\n } else {\n std::string::size_type slashPos = args.find_first_of('/');\n if (slashPos == std::string::npos || slashPos == args.size() - 1) {\n context->log(\"invalid install location: \" + args);\n context->failed();\n } else {\n std::string targetDir(args, 0, slashPos);\n std::string name(args, slashPos + 1);\n\n bool matched = false;\n BuildContext::InstallLocation location;\n\n for (int i = 0; i < BuildContext::INSTALL_LOCATION_COUNT; i++) {\n if (targetDir == BuildContext::INSTALL_LOCATION_NAMES[i]) {\n location = static_cast(i);\n matched = true;\n break;\n }\n }\n\n if (matched) {\n context->install(file, location, name);\n } else {\n context->log(\"invalid install location: \" + args);\n }\n }\n }\n } else if (command == \"passed\") {\n context->passed();\n } else {\n context->log(\"invalid command: \" + command);\n context->failed();\n }\n }\n\n void eof() {\n // Gather provisions and pass to context.\n std::vector tags;\n File* currentFile = NULL;\n\n for (ProvisionMap::iterator iter = provisions.begin(); iter != provisions.end(); ++iter) {\n if (iter->first != currentFile && !tags.empty()) {\n context->provide(currentFile, tags);\n tags.clear();\n }\n currentFile = iter->first;\n tags.push_back(iter->second);\n }\n if (!tags.empty()) {\n context->provide(currentFile, tags);\n }\n\n // Also register new triggers.\n context->addActionType(newOwned(\n executable.release(), std::move(verb), silent, std::move(triggers)));\n }\n\nprivate:\n BuildContext* context;\n OwnedPtr executable;\n OwnedPtr input; // nullable\n OwnedPtr requestStream;\n OwnedPtr responseStream;\n LineReader lineReader;\n\n std::string verb;\n bool silent;\n std::vector triggers;\n\n OwnedPtrMap knownFiles;\n\n typedef std::unordered_map CacheMap;\n CacheMap cache;\n OwnedPtrVector diskRefs;\n\n typedef std::multimap ProvisionMap;\n ProvisionMap provisions;\n\n bool findInCache(const std::string& line) {\n CacheMap::const_iterator iter = cache.find(line);\n if (iter == cache.end()) {\n return false;\n } else {\n std::string path = iter->second->path();\n responseStream->writeAll(path.data(), path.size());\n responseStream->writeAll(\"\\n\", 1);\n return true;\n }\n }\n};\n\nPromise PluginDerivedAction::start(EventManager* eventManager, BuildContext* context) {\n auto subprocess = newOwned();\n\n subprocess->addArgument(executable.get(), File::READ);\n if (file != NULL) {\n subprocess->addArgument(file->canonicalName());\n }\n\n OwnedPtr responseStream = subprocess->captureStdin();\n OwnedPtr commandStream = subprocess->captureStdout();\n OwnedPtr logStream = subprocess->captureStderr();\n\n auto subprocessWaitOp = eventManager->when(subprocess->start(eventManager))(\n [context](ProcessExitCode exitCode) {\n if (exitCode.wasSignaled() || exitCode.getExitCode() != 0) {\n context->failed();\n }\n });\n\n auto commandReader = newOwned(\n context, commandStream.release(), responseStream.release(), executable.get(), file.get());\n auto commandOp = commandReader->readAll(eventManager);\n\n OwnedPtr logger = newOwned(context, logStream.release());\n auto logOp = logger->run(eventManager);\n\n return eventManager->when(subprocessWaitOp, commandOp, logOp, subprocess, commandReader, logger)(\n [](Void, Void, Void, OwnedPtr, OwnedPtr, OwnedPtr){});\n}\n\n// =======================================================================================\n\nPluginDerivedActionFactory::PluginDerivedActionFactory(OwnedPtr executable,\n std::string&& verb,\n bool silent,\n std::vector&& triggers)\n : executable(executable.release()), silent(silent) {\n this->verb.swap(verb);\n this->triggers.swap(triggers);\n}\nPluginDerivedActionFactory::~PluginDerivedActionFactory() {}\n\nvoid PluginDerivedActionFactory::enumerateTriggerTags(\n std::back_insert_iterator > iter) {\n for (unsigned int i = 0; i < triggers.size(); i++) {\n *iter++ = triggers[i];\n }\n}\nOwnedPtr PluginDerivedActionFactory::tryMakeAction(const Tag& id, File* file) {\n return newOwned(executable.get(), verb, silent, file);\n}\n\n// =======================================================================================\n\nExecPluginActionFactory::ExecPluginActionFactory() {}\nExecPluginActionFactory::~ExecPluginActionFactory() {}\n\n// implements ActionFactory --------------------------------------------------------------\n\nvoid ExecPluginActionFactory::enumerateTriggerTags(\n std::back_insert_iterator > iter) {\n *iter++ = Tag::fromName(\"filetype:.ekam-rule\");\n}\n\nOwnedPtr ExecPluginActionFactory::tryMakeAction(const Tag& id, File* file) {\n return newOwned(file, \"learn\", false, (File*)NULL);\n}\n\n} // namespace ekam\n" }, { "path": "src/ekam/ExecPluginActionFactory.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_EKAM_EXECPLUGINACTIONFACTORY_H_\n#define KENTONSCODE_EKAM_EXECPLUGINACTIONFACTORY_H_\n\n#include \"Action.h\"\n\nnamespace ekam {\n\nclass ExecPluginActionFactory : public ActionFactory {\npublic:\n ExecPluginActionFactory();\n ~ExecPluginActionFactory();\n\n // implements ActionFactory ------------------------------------------------------------\n void enumerateTriggerTags(std::back_insert_iterator > iter);\n OwnedPtr tryMakeAction(const Tag& id, File* file);\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_EKAM_EXECPLUGINACTIONFACTORY_H_\n" }, { "path": "src/ekam/MuxDashboard.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"MuxDashboard.h\"\n\n#include \n\n#include \"base/Debug.h\"\n\nnamespace ekam {\n\nclass MuxDashboard::TaskImpl : public Dashboard::Task {\npublic:\n TaskImpl(MuxDashboard* mux, const std::string& verb, const std::string& noun, Silence silence);\n ~TaskImpl();\n\n void attach(Dashboard* dashboard);\n void detach(Dashboard* dashboard);\n\n // implements Task ---------------------------------------------------------------------\n void setState(TaskState state);\n void addOutput(const std::string& text);\n\nprivate:\n MuxDashboard* mux;\n TaskState state;\n Silence silence;\n std::string verb;\n std::string noun;\n std::string outputText;\n\n typedef OwnedPtrMap WrappedTasksMap;\n WrappedTasksMap wrappedTasks;\n\n static const size_t OUTPUT_BUFER_LIMIT = 4096 - sizeof(\"\\n...(log truncated)...\");\n};\n\nconst size_t MuxDashboard::TaskImpl::OUTPUT_BUFER_LIMIT;\n\nMuxDashboard::TaskImpl::TaskImpl(MuxDashboard* mux, const std::string& verb,\n const std::string& noun, Silence silence)\n : mux(mux), state(PENDING), silence(silence), verb(verb), noun(noun) {\n mux->tasks.insert(this);\n\n for (std::unordered_set::iterator iter = mux->wrappedDashboards.begin();\n iter != mux->wrappedDashboards.end(); ++iter) {\n wrappedTasks.add(*iter, (*iter)->beginTask(verb, noun, silence));\n }\n}\nMuxDashboard::TaskImpl::~TaskImpl() {\n mux->tasks.erase(this);\n}\n\nvoid MuxDashboard::TaskImpl::attach(Dashboard* dashboard) {\n OwnedPtr wrappedTask = dashboard->beginTask(verb, noun, silence);\n if (!outputText.empty()) {\n wrappedTask->addOutput(outputText);\n }\n if (state != PENDING) {\n wrappedTask->setState(state);\n }\n\n if (!wrappedTasks.addIfNew(dashboard, wrappedTask.release())) {\n DEBUG_ERROR << \"Tried to attach task to a dashboard to which the task was already attached.\";\n }\n}\n\nvoid MuxDashboard::TaskImpl::detach(Dashboard* dashboard) {\n if (wrappedTasks.erase(dashboard) == 0) {\n DEBUG_ERROR << \"Tried to detach task from dashboard to which it was not attached.\";\n }\n}\n\nvoid MuxDashboard::TaskImpl::setState(TaskState state) {\n if (state == PENDING || state == RUNNING) {\n outputText.clear();\n }\n\n this->state = state;\n for (WrappedTasksMap::Iterator iter(wrappedTasks); iter.next();) {\n iter.value()->setState(state);\n }\n}\n\nvoid MuxDashboard::TaskImpl::addOutput(const std::string& text) {\n if (outputText.size() < OUTPUT_BUFER_LIMIT) {\n if (outputText.size() + text.size() < OUTPUT_BUFER_LIMIT) {\n outputText.append(text);\n } else {\n outputText.append(text, 0, OUTPUT_BUFER_LIMIT - outputText.size());\n outputText.append(\"\\n...(log truncated)...\");\n }\n }\n\n for (WrappedTasksMap::Iterator iter(wrappedTasks); iter.next();) {\n iter.value()->addOutput(text);\n }\n}\n\n// =======================================================================================\n\nMuxDashboard::MuxDashboard() {}\nMuxDashboard::~MuxDashboard() {}\n\nOwnedPtr MuxDashboard::beginTask(const std::string& verb, const std::string& noun,\n Silence silence) {\n return newOwned(this, verb, noun, silence);\n}\n\nMuxDashboard::Connector::Connector(MuxDashboard* mux, Dashboard* dashboard)\n : mux(mux), dashboard(dashboard) {\n if (!mux->wrappedDashboards.insert(dashboard).second) {\n throw std::invalid_argument(\"Mux is already attached to this dashboard.\");\n }\n\n for (std::unordered_set::iterator iter = mux->tasks.begin();\n iter != mux->tasks.end(); ++iter) {\n (*iter)->attach(dashboard);\n }\n}\n\nMuxDashboard::Connector::~Connector() {\n if (mux->wrappedDashboards.erase(dashboard) == 0) {\n DEBUG_ERROR << \"Deleting MuxDashboard connection that was never made?\";\n }\n\n for (std::unordered_set::iterator iter = mux->tasks.begin();\n iter != mux->tasks.end(); ++iter) {\n (*iter)->detach(dashboard);\n }\n}\n\n} // namespace ekam\n" }, { "path": "src/ekam/MuxDashboard.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_EKAM_MUXDASHBOARD_H_\n#define KENTONSCODE_EKAM_MUXDASHBOARD_H_\n\n#include \n\n#include \"Dashboard.h\"\n\nnamespace ekam {\n\nclass MuxDashboard : public Dashboard {\npublic:\n MuxDashboard();\n ~MuxDashboard();\n\n class Connector {\n public:\n Connector(MuxDashboard* mux, Dashboard* dashboard);\n ~Connector();\n\n private:\n MuxDashboard* mux;\n Dashboard* dashboard;\n };\n\n // implements Dashboard ----------------------------------------------------------------\n OwnedPtr beginTask(const std::string& verb, const std::string& noun, Silence silence);\n\nprivate:\n class TaskImpl;\n\n std::unordered_set tasks;\n std::unordered_set wrappedDashboards;\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_EKAM_MUXDASHBOARD_H_\n" }, { "path": "src/ekam/ProtoDashboard.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"ProtoDashboard.h\"\n\n#include \n#include \n#include \n#include \n#include \n\n#include \"dashboard.capnp.h\"\n#include \"os/Socket.h\"\n#include \"MuxDashboard.h\"\n\nnamespace ekam {\n\nclass ProtoDashboard::TaskImpl : public Dashboard::Task {\npublic:\n TaskImpl(int id, const std::string& verb, const std::string& noun,\n Silence silence, WriteBuffer* output);\n ~TaskImpl();\n\n // implements Task ---------------------------------------------------------------------\n void setState(TaskState state);\n void addOutput(const std::string& text);\n\nprivate:\n int id;\n WriteBuffer* output;\n\n static const proto::TaskUpdate::State STATE_CODES[];\n};\n\nconst proto::TaskUpdate::State ProtoDashboard::TaskImpl::STATE_CODES[] = {\n proto::TaskUpdate::State::PENDING,\n proto::TaskUpdate::State::RUNNING,\n proto::TaskUpdate::State::DONE ,\n proto::TaskUpdate::State::PASSED ,\n proto::TaskUpdate::State::FAILED ,\n proto::TaskUpdate::State::BLOCKED\n};\n\nProtoDashboard::TaskImpl::TaskImpl(int id, const std::string& verb, const std::string& noun,\n Silence silence, WriteBuffer* output)\n : id(id), output(output) {\n capnp::MallocMessageBuilder message;\n proto::TaskUpdate::Builder update = message.getRoot();\n update.setId(id);\n update.setState(proto::TaskUpdate::State::PENDING);\n update.setVerb(verb);\n update.setNoun(noun);\n update.setSilent(silence == SILENT);\n output->write(message.getSegmentsForOutput());\n}\n\nProtoDashboard::TaskImpl::~TaskImpl() {\n capnp::MallocMessageBuilder message;\n proto::TaskUpdate::Builder update = message.getRoot();\n update.setId(id);\n update.setState(proto::TaskUpdate::State::DELETED);\n output->write(message.getSegmentsForOutput());\n}\n\nvoid ProtoDashboard::TaskImpl::setState(TaskState state) {\n capnp::MallocMessageBuilder message;\n proto::TaskUpdate::Builder update = message.getRoot();\n update.setId(id);\n update.setState(STATE_CODES[state]);\n output->write(message.getSegmentsForOutput());\n}\n\nvoid ProtoDashboard::TaskImpl::addOutput(const std::string& text) {\n capnp::MallocMessageBuilder message;\n proto::TaskUpdate::Builder update = message.getRoot();\n update.setId(id);\n update.setLog(text);\n output->write(message.getSegmentsForOutput());\n}\n\n// =======================================================================================\n\nProtoDashboard::ProtoDashboard(EventManager* eventManager, OwnedPtr stream)\n : idCounter(0),\n writeBuffer(eventManager, stream.release()) {\n capnp::MallocMessageBuilder message;\n proto::Header::Builder header = message.getRoot();\n char* cwd = get_current_dir_name();\n header.setProjectRoot(cwd);\n free(cwd);\n writeBuffer.write(message.getSegmentsForOutput());\n}\nProtoDashboard::~ProtoDashboard() {}\n\nOwnedPtr ProtoDashboard::beginTask(\n const std::string& verb, const std::string& noun, Silence silence) {\n return newOwned(++idCounter, verb, noun, silence, &writeBuffer);\n}\n\n// =======================================================================================\n\nProtoDashboard::WriteBuffer::WriteBuffer(EventManager* eventManager,\n OwnedPtr stream)\n : eventManager(eventManager), stream(stream.release()),\n ioWatcher(eventManager->watchFd(this->stream->getHandle()->get())),\n offset(0), disconnectFulfiller(NULL) {}\nProtoDashboard::WriteBuffer::~WriteBuffer() {}\n\nvoid ProtoDashboard::WriteBuffer::write(kj::ArrayPtr> message) {\n if (stream == NULL) {\n // Already disconnected.\n return;\n }\n\n messages.push(capnp::messageToFlatArray(message));\n\n ready();\n}\n\nvoid ProtoDashboard::WriteBuffer::ready() {\n try {\n while (!messages.empty()) {\n const kj::Array& message = messages.front();\n while (offset < message.size()) {\n offset += stream->write(message.asBytes().begin() + offset,\n message.asBytes().size() - offset);\n }\n offset = 0;\n messages.pop();\n }\n } catch (const OsError& error) {\n if (error.getErrorNumber() == EAGAIN) {\n // Ran out of kernel buffer space. Wait until writable again.\n waitWritablePromise = eventManager->when(ioWatcher->onWritable())(\n [this](Void) {\n ready();\n });\n } else {\n stream.clear();\n\n if (disconnectFulfiller != NULL) {\n disconnectFulfiller->disconnected();\n }\n }\n }\n}\n\n// =======================================================================================\n\nProtoDashboard::WriteBuffer::DisconnectFulfiller::DisconnectFulfiller(Callback* callback,\n WriteBuffer* writeBuffer)\n : callback(callback), writeBuffer(writeBuffer) {\n if (writeBuffer->disconnectFulfiller != NULL) {\n throw std::logic_error(\"Can only register one disconnect callback at a time.\");\n }\n writeBuffer->disconnectFulfiller = this;\n}\n\nProtoDashboard::WriteBuffer::DisconnectFulfiller::~DisconnectFulfiller() {\n assert(writeBuffer->disconnectFulfiller == this);\n writeBuffer->disconnectFulfiller = NULL;\n}\n\nPromise ProtoDashboard::onDisconnect() {\n return writeBuffer.onDisconnect();\n}\n\nPromise ProtoDashboard::WriteBuffer::onDisconnect() {\n return newPromise(this);\n}\n\n// =======================================================================================\n\nclass NetworkAcceptingDashboard : public Dashboard {\npublic:\n NetworkAcceptingDashboard(EventManager* eventManager, const std::string& address,\n OwnedPtr baseDashboard)\n : eventManager(eventManager),\n base(baseDashboard.release()),\n baseConnector(newOwned(&mux, base.get())),\n socket(newOwned(eventManager, address)),\n acceptOp(doAccept()) {}\n ~NetworkAcceptingDashboard() {}\n\n Promise doAccept() {\n return eventManager->when(socket->accept())(\n [this](OwnedPtr stream){\n accepted(stream.release());\n return doAccept();\n });\n }\n\n void accepted(OwnedPtr stream);\n\n // implements Dashboard ----------------------------------------------------------------\n OwnedPtr beginTask(const std::string& verb, const std::string& noun, Silence silence) {\n return mux.beginTask(verb, noun, silence);\n }\n\nprivate:\n EventManager* eventManager;\n OwnedPtr base;\n MuxDashboard mux;\n OwnedPtr baseConnector;\n OwnedPtr socket;\n Promise acceptOp;\n\n class ConnectedProtoDashboard {\n public:\n ConnectedProtoDashboard(NetworkAcceptingDashboard* owner, EventManager* eventManager,\n OwnedPtr stream)\n : protoDashboard(eventManager, stream.release()),\n connector(newOwned(&owner->mux, &protoDashboard)) {\n disconnectPromise = eventManager->when(protoDashboard.onDisconnect())(\n [this, owner](Void) {\n connector.clear();\n owner->connectedDashboards.erase(this);\n });\n }\n ~ConnectedProtoDashboard() {}\n\n private:\n ProtoDashboard protoDashboard;\n OwnedPtr connector;\n Promise disconnectPromise;\n };\n OwnedPtrMap connectedDashboards;\n};\n\nvoid NetworkAcceptingDashboard::accepted(OwnedPtr stream) {\n auto connectedDashboard = newOwned(this, eventManager, stream.release());\n auto key = connectedDashboard.get(); // cannot inline due to undefined evaluation order\n connectedDashboards.add(key, connectedDashboard.release());\n}\n\nOwnedPtr initNetworkDashboard(EventManager* eventManager, const std::string& address,\n OwnedPtr dashboardToWrap) {\n return newOwned(eventManager, address, dashboardToWrap.release());\n}\n\n} // namespace ekam\n" }, { "path": "src/ekam/ProtoDashboard.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_EKAM_PROTODASHBOARD_H_\n#define KENTONSCODE_EKAM_PROTODASHBOARD_H_\n\n#include \n#include \n#include \n\n#include \"Dashboard.h\"\n#include \"os/ByteStream.h\"\n#include \"os/EventManager.h\"\n\nnamespace ekam {\n\nclass ProtoDashboard : public Dashboard {\npublic:\n ProtoDashboard(EventManager* eventManager, OwnedPtr stream);\n ~ProtoDashboard();\n\n Promise onDisconnect();\n\n // implements Dashboard ----------------------------------------------------------------\n OwnedPtr beginTask(const std::string& verb, const std::string& noun, Silence silence);\n\nprivate:\n class TaskImpl;\n\n class WriteBuffer {\n public:\n WriteBuffer(EventManager* eventManager, OwnedPtr stream);\n ~WriteBuffer();\n\n void write(kj::ArrayPtr> data);\n Promise onDisconnect();\n\n private:\n EventManager* eventManager;\n OwnedPtr stream;\n OwnedPtr ioWatcher;\n std::queue> messages;\n std::string::size_type offset;\n Promise waitWritablePromise;\n\n class DisconnectFulfiller : public PromiseFulfiller {\n public:\n DisconnectFulfiller(Callback* callback, WriteBuffer* writeBuffer);\n ~DisconnectFulfiller();\n\n void disconnected() { callback->fulfill(); }\n\n private:\n Callback* callback;\n WriteBuffer* writeBuffer;\n };\n DisconnectFulfiller* disconnectFulfiller;\n\n void ready();\n };\n\n int idCounter;\n WriteBuffer writeBuffer;\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_EKAM_PROTODASHBOARD_H_\n" }, { "path": "src/ekam/SimpleDashboard.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"SimpleDashboard.h\"\n\nnamespace ekam {\n\nclass SimpleDashboard::TaskImpl : public Dashboard::Task {\npublic:\n TaskImpl(const std::string& verb, const std::string& noun, Silence silence, FILE* outputStream);\n ~TaskImpl();\n\n // implements Task ---------------------------------------------------------------------\n void setState(TaskState state);\n void addOutput(const std::string& text);\n\nprivate:\n TaskState state;\n Silence silence;\n std::string verb;\n std::string noun;\n std::string outputText;\n FILE* outputStream;\n\n static const char* const STATE_NAMES[];\n};\n\nconst char* const SimpleDashboard::TaskImpl::STATE_NAMES[] = {\n \"PENDING\",\n \"RUNNING\",\n \"DONE \",\n \"PASSED \",\n \"FAILED \",\n \"BLOCKED\"\n};\n\nSimpleDashboard::TaskImpl::TaskImpl(const std::string& verb, const std::string& noun,\n Silence silence, FILE* outputStream)\n : state(PENDING), silence(silence), verb(verb), noun(noun), outputStream(outputStream) {}\nSimpleDashboard::TaskImpl::~TaskImpl() {}\n\nvoid SimpleDashboard::TaskImpl::setState(TaskState state) {\n // If state was previously BLOCKED, and we managed to un-block, then we don't care about the\n // reason why we were blocked, so clear the text.\n if (this->state == BLOCKED && (state == PENDING || state == RUNNING)) {\n outputText.clear();\n }\n this->state = state;\n\n bool writeOutput = !outputText.empty() && state != BLOCKED;\n\n if (silence != SILENT || writeOutput) {\n // Write status update.\n fprintf(outputStream, \"[%s] %s: %s\\n\", STATE_NAMES[state], verb.c_str(), noun.c_str());\n\n // Write any output we have buffered, unless new state is BLOCKED in which case we save the\n // output for later.\n if (writeOutput) {\n fwrite(outputText.c_str(), sizeof(char), outputText.size(), outputStream);\n if (outputText[outputText.size() - 1] != '\\n') {\n fputc('\\n', outputStream);\n }\n outputText.clear();\n }\n\n fflush(outputStream);\n }\n}\n\nvoid SimpleDashboard::TaskImpl::addOutput(const std::string& text) {\n outputText.append(text);\n}\n\n// =======================================================================================\n\nSimpleDashboard::SimpleDashboard(FILE* outputStream) : outputStream(outputStream) {}\nSimpleDashboard::~SimpleDashboard() {}\n\nOwnedPtr SimpleDashboard::beginTask(\n const std::string& verb, const std::string& noun, Silence silence) {\n return newOwned(verb, noun, silence, outputStream);\n}\n\n} // namespace ekam\n" }, { "path": "src/ekam/SimpleDashboard.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_EKAM_SIMPLEDASHBOARD_H_\n#define KENTONSCODE_EKAM_SIMPLEDASHBOARD_H_\n\n#include \n\n#include \"Dashboard.h\"\n\nnamespace ekam {\n\nclass SimpleDashboard : public Dashboard {\npublic:\n SimpleDashboard(FILE* outputStream);\n ~SimpleDashboard();\n\n // implements Dashboard ----------------------------------------------------------------\n OwnedPtr beginTask(const std::string& verb, const std::string& noun, Silence silence);\n\nprivate:\n class TaskImpl;\n\n FILE* outputStream;\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_EKAM_SIMPLEDASHBOARD_H_\n" }, { "path": "src/ekam/Tag.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"Tag.h\"\n#include \"base/Debug.h\"\n\nnamespace ekam {\n\nnamespace {\n\nstd::string canonicalizePath(const std::string& path) {\n std::vector parts;\n\n std::string::size_type pos = 0;\n while (pos != std::string::npos) {\n std::string::size_type slashPos = path.find_first_of('/', pos);\n\n std::string part;\n if (slashPos == std::string::npos) {\n part.assign(path, pos, path.size() - pos);\n pos = slashPos;\n } else {\n part.assign(path, pos, slashPos - pos);\n pos = path.find_first_not_of('/', slashPos);\n }\n\n if (part.empty() || part == \".\") {\n // skip\n } else if (part == \"..\") {\n if (parts.empty()) {\n // ignore\n } else {\n parts.pop_back();\n }\n } else {\n parts.push_back(part);\n }\n }\n\n std::string result;\n result.reserve(path.size());\n for (unsigned int i = 0; i < parts.size(); i++) {\n if (i > 0) {\n result.push_back('/');\n }\n result.append(parts[i]);\n }\n\n return result;\n}\n\n} // namespace\n\nconst Tag Tag::DEFAULT_TAG = Tag::fromName(\"file:*\");\n\nTag Tag::fromFile(const std::string& path) {\n return fromName(\"file:\" + canonicalizePath(path));\n}\n\n} // namespace ekam\n" }, { "path": "src/ekam/Tag.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_EKAM_TAG_H_\n#define KENTONSCODE_EKAM_TAG_H_\n\n#include \n#include \n#include \n#include \n\n#include \"base/Hash.h\"\n\nnamespace ekam {\n\nclass File;\n\nclass Tag {\npublic:\n Tag() {}\n\n // Every file has this tag.\n static const Tag DEFAULT_TAG;\n\n static inline Tag fromName(const std::string& name) {\n return Tag(name);\n }\n\n static Tag fromFile(const std::string& path);\n\n inline std::string toString() { return hash.toString(); }\n\n inline bool operator==(const Tag& other) const { return hash == other.hash; }\n inline bool operator!=(const Tag& other) const { return hash != other.hash; }\n inline bool operator< (const Tag& other) const { return hash < other.hash; }\n inline bool operator> (const Tag& other) const { return hash > other.hash; }\n inline bool operator<=(const Tag& other) const { return hash <= other.hash; }\n inline bool operator>=(const Tag& other) const { return hash >= other.hash; }\n\n class HashFunc {\n public:\n inline size_t operator()(const Tag& id) const {\n return inner(id.hash);\n }\n\n private:\n Hash::StlHashFunc inner;\n };\n\nprivate:\n Hash hash;\n\n#ifdef EXTRA_DEBUG\n std::string name;\n inline explicit Tag(const std::string& name) : hash(Hash::of(name)), name(name) {}\n#else\n inline explicit Tag(const std::string& name) : hash(Hash::of(name)) {}\n#endif\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_EKAM_TAG_H_\n" }, { "path": "src/ekam/dashboard.capnp", "content": "# Ekam Build System\n# Author: Kenton Varda (kenton@sandstorm.io)\n# Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n@0xa610fcb94ceea9cc;\n\n$import \"/capnp/c++.capnp\".namespace(\"ekam::proto\");\n\nstruct Header {\n # The first message sent over the stream is the header.\n\n projectRoot @0 :Text;\n # The directory where ekam was run, containing \"src\", \"tmp\", etc.\n}\n\nstruct TaskUpdate {\n # All subsequent messages are TaskUpdates.\n\n id @0 :UInt32;\n\n state @1 :State = unchanged;\n enum State {\n unchanged @0;\n deleted @1;\n pending @2;\n running @3;\n done @4;\n passed @5;\n failed @6;\n blocked @7;\n }\n\n verb @2 :Text;\n noun @3 :Text;\n silent @4 :Bool;\n log @5 :Text;\n}\n" }, { "path": "src/ekam/ekam-client.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"dashboard.capnp.h\"\n#include \n#include \n#include \n#include \n#include \n#include \n#include \"ConsoleDashboard.h\"\n#include \"base/OwnedPtr.h\"\n\nnamespace ekam {\n\nvoid dump(proto::TaskUpdate::Reader message) {\n using std::cerr;\n using std::cout;\n using std::endl;\n\n cout << \"================================================================================\\n\";\n\n cout << message.getId() << \":\";\n if (message.getState() != proto::TaskUpdate::State::UNCHANGED) {\n cout << \" \" << kj::str(message.getState()).cStr();\n }\n if (message.hasVerb()) {\n cout << \" \" << message.getVerb().cStr();\n }\n if (message.hasNoun()) {\n cout << \" \" << message.getNoun().cStr();\n }\n if (message.getSilent()) {\n cout << \" (silent)\";\n }\n cout << '\\n';\n\n if (message.hasLog()) {\n auto log = message.getLog();\n cout << log.cStr();\n if (!log.endsWith(\"\\n\")) {\n cout << '\\n';\n }\n }\n\n cout.flush();\n}\n\nDashboard::TaskState toDashboardState(proto::TaskUpdate::State state) {\n switch (state) {\n case proto::TaskUpdate::State::PENDING:\n return Dashboard::PENDING;\n case proto::TaskUpdate::State::RUNNING:\n return Dashboard::RUNNING;\n case proto::TaskUpdate::State::DONE:\n return Dashboard::DONE;\n case proto::TaskUpdate::State::PASSED:\n return Dashboard::PASSED;\n case proto::TaskUpdate::State::FAILED:\n return Dashboard::FAILED;\n case proto::TaskUpdate::State::BLOCKED:\n return Dashboard::BLOCKED;\n default:\n throw std::invalid_argument(kj::str(\"Invalid state: \", state).cStr());\n }\n}\n\nint main(int argc, char* argv[]) {\n int maxDisplayedLogLines = 30;\n \n for (int i = 1; i < argc; i++) {\n if (strcmp(argv[i], \"-l\") == 0) {\n char* endptr;\n if (i + 1 >= argc ||\n (maxDisplayedLogLines = strtoul(argv[++i], &endptr, 0), *endptr != '\\0')) {\n fprintf(stderr, \"Expected number after -l.\\n\");\n return 1;\n }\n } else if (strcmp(argv[i], \"-h\") == 0 || strcmp(argv[i], \"--help\") == 0) {\n printf(\n \"usage: nc | %s [-l ]\\n\"\n \"\\n\"\n \"Connect to Ekam process at and display build status.\\n\"\n \"\\n\"\n \"options:\\n\"\n \" -l Set max number of log lines to display per action. This is\\n\"\n \" kept relatively short by default because it makes the build\\n\"\n \" output noisy, but you may need to increase it if you need\\n\"\n \" to see more of a particular error log.\\n\",\n argv[0]);\n return 0;\n } else {\n fprintf(stderr, \"Unknown option: %s\\n\", argv[i]);\n return 1;\n }\n }\n\n kj::FdInputStream rawInput(STDIN_FILENO);\n kj::BufferedInputStreamWrapper bufferedInput(rawInput);\n\n {\n capnp::InputStreamMessageReader message(bufferedInput);\n proto::Header::Reader header = message.getRoot();\n printf(\"Project root: %s\\n\", header.getProjectRoot().cStr());\n }\n\n ConsoleDashboard dashboard(stdout, maxDisplayedLogLines);\n OwnedPtrMap tasks;\n\n while (bufferedInput.tryGetReadBuffer() != nullptr) {\n capnp::InputStreamMessageReader messageReader(bufferedInput);\n proto::TaskUpdate::Reader message = messageReader.getRoot();\n\n if (message.getState() == proto::TaskUpdate::State::DELETED) {\n tasks.erase(message.getId());\n } else if (Dashboard::Task* task = tasks.get(message.getId())) {\n if (message.hasLog()) {\n task->addOutput(message.getLog());\n }\n if (message.getState() != proto::TaskUpdate::State::UNCHANGED) {\n task->setState(toDashboardState(message.getState()));\n }\n } else {\n OwnedPtr newTask = dashboard.beginTask(\n message.getVerb(), message.getNoun(),\n message.getSilent() ? Dashboard::SILENT : Dashboard::NORMAL);\n if (message.hasLog()) {\n newTask->addOutput(message.getLog());\n }\n if (message.getState() != proto::TaskUpdate::State::UNCHANGED &&\n message.getState() != proto::TaskUpdate::State::PENDING) {\n newTask->setState(toDashboardState(message.getState()));\n }\n tasks.add(message.getId(), newTask.release());\n }\n }\n\n return 0;\n}\n\n} // namespace ekam\n\nint main(int argc, char* argv[]) {\n return ekam::main(argc, argv);\n}\n" }, { "path": "src/ekam/ekam-langserve.c++", "content": "#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n\nnamespace ekam {\n\ntypedef unsigned int uint;\n\nclass AsyncIoStreamPair final: public kj::AsyncIoStream {\npublic:\n AsyncIoStreamPair(kj::Own input,\n kj::Own output)\n : input(kj::mv(input)), output(kj::mv(output)) {}\n\n kj::Promise read(void* buffer, size_t minBytes, size_t maxBytes) override {\n return input->read(buffer, minBytes, maxBytes);\n }\n kj::Promise tryRead(void* buffer, size_t minBytes, size_t maxBytes) override {\n return input->tryRead(buffer, minBytes, maxBytes);\n }\n kj::Maybe tryGetLength() override {\n return input->tryGetLength();\n }\n kj::Promise pumpTo(\n AsyncOutputStream& output, uint64_t amount = kj::maxValue) override {\n return input->pumpTo(output, amount);\n }\n\n kj::Promise write(const void* buffer, size_t size) override {\n return output->write(buffer, size);\n }\n kj::Promise write(kj::ArrayPtr> pieces) override {\n return output->write(pieces);\n }\n kj::Maybe> tryPumpFrom(\n AsyncInputStream& input, uint64_t amount = kj::maxValue) override {\n return output->tryPumpFrom(input, amount);\n }\n kj::Promise whenWriteDisconnected() override {\n return output->whenWriteDisconnected();\n }\n\n void shutdownWrite() override {\n output = nullptr;\n }\n void abortRead() override {\n input = nullptr;\n }\n\nprivate:\n kj::Own input;\n kj::Own output;\n};\n\nclass SourceFile;\n\nstruct Message {\n SourceFile* file;\n uint line;\n uint column;\n uint endColumn;\n kj::String text;\n\n inline bool operator==(const Message& other) const {\n return file == other.file &&\n line == other.line &&\n column == other.column &&\n endColumn == other.endColumn &&\n text == other.text;\n }\n inline uint hashCode() const {\n return kj::hashCode(file, line, column, endColumn, text);\n }\n\n void exportRange(lsp::Range::Builder range) {\n auto start = range.initStart();\n start.setLine(line == 0 ? 0 : line - 1);\n start.setCharacter(column == 0 ? 0 : column - 1);\n\n auto end = range.initEnd();\n end.setLine(line == 0 ? 0 : line - 1);\n end.setCharacter(endColumn == 0 ? (column == 0 ? 65536 : column - 1) : endColumn - 1);\n }\n};\n\nenum class Severity: uint8_t {\n NOTE = 0,\n ERROR = 1,\n WARNING = 2,\n INFO = 3,\n HINT = 4\n};\n\nstruct Diagnostic {\n Severity severity;\n Message message;\n kj::Vector notes;\n\n inline bool operator==(const Diagnostic& other) const {\n return severity == other.severity &&\n message == other.message;\n }\n inline uint hashCode() const {\n return kj::hashCode(static_cast(severity), message);\n }\n};\n\nclass SourceFile;\n\nclass DirtySet {\npublic:\n void add(SourceFile* file) {\n dirty.upsert(file, [](auto...) {});\n KJ_IF_MAYBE(f, fulfiller) {\n f->get()->fulfill();\n fulfiller = nullptr;\n }\n }\n template \n void forEach(Func&& func) {\n for (auto file: dirty) {\n func(*file);\n }\n dirty.clear();\n }\n\n kj::Maybe> whenNonEmpty() {\n KJ_REQUIRE(fulfiller == nullptr, \"can only call whenNonEmpty() once at a time\");\n if (dirty.size() > 0) {\n return kj::Promise(kj::READY_NOW);\n } else if (isShutdown) {\n return nullptr;\n } else {\n auto paf = kj::newPromiseAndFulfiller();\n fulfiller = kj::mv(paf.fulfiller);\n return kj::mv(paf.promise);\n }\n }\n\n void shutdown() {\n // Make it so calling whenNotEmpty() when empty returns null rather than blocking.\n KJ_IF_MAYBE(f, fulfiller) {\n f->get()->fulfill();\n fulfiller = nullptr;\n }\n isShutdown = true;\n }\n\nprivate:\n kj::HashSet dirty;\n kj::Maybe>> fulfiller;\n bool isShutdown = false;\n};\n\nclass SourceFile {\npublic:\n SourceFile(DirtySet& dirtySet, kj::String realPath)\n : dirtySet(dirtySet), realPath(kj::mv(realPath)) {}\n inline kj::StringPtr getRealPath() { return realPath; }\n\n void markDirty() {\n dirtySet.add(this);\n }\n\n void markStale() {\n // We were informed the file has changed, which potentially invalidates all diagnostics.\n // Mark them all stale.\n bool dirty = false;\n for (auto& diagnostic: diagnostics) {\n if (!diagnostic.stale) {\n diagnostic.stale = true;\n dirty = true;\n }\n }\n if (dirty) {\n markDirty();\n }\n }\n\n Diagnostic& add(Diagnostic&& diagnostic) {\n auto& result = diagnostics.findOrCreate(diagnostic, [&]() {\n dirtySet.add(this);\n return DiagnosticEntry { kj::heap(kj::mv(diagnostic)), 0 };\n });\n ++result.refcount;\n result.stale = false;\n return *result.diagnostic;\n }\n\n void remove(Diagnostic& diagnostic) {\n auto& entry = KJ_ASSERT_NONNULL(diagnostics.find(diagnostic));\n KJ_ASSERT(entry.diagnostic.get() == &diagnostic);\n if (--entry.refcount == 0) {\n diagnostics.erase(entry);\n dirtySet.add(this);\n }\n }\n\n void exportDiagnostics(kj::StringPtr uriPrefix,\n lsp::LanguageClient::PublishDiagnosticsParams::Builder builder) {\n builder.setUri(kj::str(uriPrefix, realPath));\n\n size_t count = 0;\n for (auto& diagnostic: diagnostics) {\n if (!diagnostic.stale) ++count;\n }\n\n auto list = builder.initDiagnostics(count);\n auto iter = diagnostics.begin();\n for (auto out: list) {\n while (iter->stale) ++iter;\n auto& in = *(iter++)->diagnostic;\n\n out.setSeverity((uint)in.severity);\n in.message.exportRange(out.initRange());\n out.setMessage(in.message.text);\n out.setSource(\"ekam\");\n\n if (!in.notes.empty()) {\n auto notes = out.initRelatedInformation(in.notes.size());\n for (auto i: kj::indices(in.notes)) {\n auto outNote = notes[i];\n auto& inNote = in.notes[i];\n\n auto loc = outNote.initLocation();\n loc.setUri(kj::str(uriPrefix, inNote.file->realPath));\n inNote.exportRange(loc.initRange());\n outNote.setMessage(inNote.text);\n }\n }\n }\n }\n\nprivate:\n DirtySet& dirtySet;\n kj::String realPath;\n\n struct DiagnosticEntry {\n kj::Own diagnostic;\n uint refcount;\n bool stale = false;\n\n bool operator==(const Diagnostic& other) const { return *diagnostic == other; }\n bool operator==(const DiagnosticEntry& other) const { return *diagnostic == *other.diagnostic; }\n uint hashCode() const { return kj::hashCode(*diagnostic); }\n };\n kj::HashSet diagnostics;\n};\n\nclass SourceFileSet {\npublic:\n SourceFileSet(const kj::ReadableDirectory& projectHome, DirtySet& dirtySet)\n : projectHome(projectHome), dirtySet(dirtySet) {}\n\n kj::Maybe get(kj::StringPtr name) {\n static constexpr kj::StringPtr STRIP_PREFIXES[] = {\n \"/ekam-provider/canonical/\"_kj,\n \"/ekam-provider/c++header/\"_kj\n };\n for (auto prefix: STRIP_PREFIXES) {\n if (name.startsWith(prefix)) {\n name = name.slice(prefix.size());\n break;\n }\n }\n\n auto deref = [](kj::Maybe>& maybe) {\n return maybe.map([](kj::Own& own) -> SourceFile& { return *own; });\n };\n\n KJ_IF_MAYBE(existing, files.find(name)) {\n return deref(*existing);\n }\n\n if (name.startsWith(\"/\")) {\n // Absolute path, probably a header or something.\n return insert(name, kj::heap(dirtySet, kj::str(name)));\n } else {\n // Look for the file under `src` and `tmp`.\n auto canonical = kj::Path::parse(name);\n static constexpr kj::StringPtr DIRS[] = {\"src\"_kj, \"tmp\"_kj};\n for (auto dir: DIRS) {\n auto path = kj::Path({dir}).append(canonical);\n if (projectHome.exists(path)) {\n // Found it.\n return insert(name, kj::heap(dirtySet,\n expandSymlinks(kj::mv(path)).toString()));\n }\n }\n\n if (projectHome.exists(canonical)) {\n // Maybe it was already non-canonical.\n return insert(name, kj::heap(dirtySet,\n expandSymlinks(kj::mv(canonical)).toString()));\n }\n\n // This path doesn't appear to be a file. Cache this fact.\n files.insert(kj::str(name), nullptr);\n return nullptr;\n }\n }\n\n kj::Maybe findByRealPath(kj::StringPtr path) {\n return filesByPath.find(path).map([](SourceFile* f) -> SourceFile& { return *f; });\n }\n\n template \n void forEach(Func&& func) {\n for (auto& entry: files) {\n KJ_IF_MAYBE(f, entry.value) {\n func(**f);\n }\n }\n }\n\nprivate:\n const kj::ReadableDirectory& projectHome;\n DirtySet& dirtySet;\n kj::HashMap>> files;\n kj::HashMap filesByPath;\n\n SourceFile& insert(kj::StringPtr name, kj::Own file) {\n auto& result = *file;\n auto& realPathEntry = filesByPath.insert(file->getRealPath(), file.get());\n KJ_ON_SCOPE_FAILURE(filesByPath.erase(realPathEntry));\n files.insert(kj::str(name), kj::mv(file));\n return result;\n }\n\n kj::Path expandSymlinks(kj::Path path) {\n retry:\n for (size_t i = 1; i < path.size(); i++) {\n auto parent = path.slice(0, i);\n KJ_IF_MAYBE(link, projectHome.tryReadlink(parent)) {\n if (!link->startsWith(\"/\")) {\n try {\n path = parent.slice(0, i-1).eval(*link).append(path.slice(i, path.size()));\n goto retry;\n } catch (const kj::Exception& e) {\n KJ_LOG(WARNING, \"bad symlink\", *link, e.getDescription());\n }\n }\n }\n }\n\n return kj::mv(path);\n }\n};\n\nkj::Maybe tryConsumeNumberColon(kj::StringPtr& str) {\n if (str.startsWith(\":\")) return nullptr;\n\n for (size_t i = 0; i < str.size(); i++) {\n if (str[i] == ':') {\n uint result = strtoul(str.cStr(), nullptr, 10);\n str = str.slice(i + 1);\n return result;\n } else if (str[i] < '0' || str[i] > '9') {\n return nullptr;\n }\n }\n\n return nullptr;\n}\n\nstruct ColumnRange {\n uint start;\n uint end;\n};\n\nkj::Maybe tryConsumeRangeColon(kj::StringPtr& str) {\n if (str.startsWith(\":\")) return nullptr;\n\n kj::Maybe start;\n size_t startPos = 0;\n for (size_t i = 0; i < str.size(); i++) {\n if (str[i] == ':') {\n uint result = strtoul(str.cStr() + startPos, nullptr, 10);\n str = str.slice(i + 1);\n KJ_IF_MAYBE(s, start) {\n return ColumnRange { *s, result };\n } else {\n return ColumnRange { result, result };\n }\n } else if (str[i] == '-' && start == nullptr) {\n start = strtoul(str.cStr(), nullptr, 10);\n startPos = i + 1;\n } else if (str[i] < '0' || str[i] > '9') {\n return nullptr;\n }\n }\n\n return nullptr;\n}\n\nvoid trimLeadingSpace(kj::StringPtr& str) {\n while (str.startsWith(\" \")) { str = str.slice(1); }\n}\n\nSeverity consumeSeverity(kj::StringPtr& line) {\n if (line.startsWith(\"note:\")) {\n line = line.slice(5);\n trimLeadingSpace(line);\n return Severity::NOTE;\n } else {\n struct SeverityPrefix {\n kj::StringPtr prefix;\n Severity severity;\n };\n static constexpr SeverityPrefix SEVERITY_PREFIXES[] = {\n { \"error:\"_kj, Severity::ERROR },\n { \"warning:\"_kj, Severity::WARNING },\n { \"warn:\"_kj, Severity::WARNING },\n { \"info:\"_kj, Severity::INFO },\n { \"hint:\"_kj, Severity::HINT },\n };\n\n for (auto prefix: SEVERITY_PREFIXES) {\n if (line.startsWith(prefix.prefix)) {\n line = line.slice(prefix.prefix.size());\n trimLeadingSpace(line);\n return prefix.severity;\n }\n }\n return Severity::ERROR;\n }\n}\n\nclass Task {\npublic:\n Task(proto::TaskUpdate::Reader update, SourceFileSet& files) {}\n ~Task() noexcept(false) {\n clearDiagnostics();\n }\n\n void update(proto::TaskUpdate::Reader update, SourceFileSet& files) {\n // Invalidate log when the task is deleted or it is re-running or scheduled to re-run.\n if (update.getState() == proto::TaskUpdate::State::PENDING ||\n update.getState() == proto::TaskUpdate::State::RUNNING ||\n update.getState() == proto::TaskUpdate::State::DELETED) {\n clearDiagnostics();\n }\n\n kj::StringPtr log = update.getLog();\n\n kj::String ownLog;\n if (leftoverLog.size() > 0) {\n ownLog = kj::str(leftoverLog, log);\n log = ownLog;\n }\n\n for (;;) {\n kj::StringPtr line;\n kj::String ownLine;\n KJ_IF_MAYBE(eol, log.findFirst('\\n')) {\n ownLine = kj::str(log.slice(0, *eol));\n line = ownLine;\n log = log.slice(*eol + 1);\n } else {\n leftoverLog = kj::str(log);\n break;\n }\n\n trimLeadingSpace(line);\n if (line == nullptr) continue;\n\n static constexpr kj::StringPtr IGNORE_PREFIXES[] = {\n \"In file included from \"_kj\n };\n bool ignore = false;\n for (auto prefix: IGNORE_PREFIXES) {\n if (line.startsWith(prefix)) {\n ignore = true;\n break;\n }\n }\n if (ignore) continue;\n\n static constexpr kj::StringPtr STRIP_PREFIXES[] = {\n // Linker errors start with this.\n \"/usr/bin/ld: \"_kj\n };\n for (auto prefix: STRIP_PREFIXES) {\n if (line.startsWith(prefix)) {\n line = line.slice(prefix.size());\n }\n }\n\n size_t spaceAt = line.findFirst(' ').orDefault(line.size());\n if (line[spaceAt - 1] != ':') {\n // No file:line:column: to parse... skip.\n continue;\n }\n\n // parse file:line:column:\n size_t pos = KJ_ASSERT_NONNULL(line.findFirst(':'));\n auto filename = kj::str(line.slice(0, pos));\n KJ_IF_MAYBE(file, files.get(filename)) {\n line = line.slice(pos + 1);\n\n kj::Maybe lineNo = tryConsumeNumberColon(line);\n kj::Maybe columnRange = tryConsumeRangeColon(line);\n\n trimLeadingSpace(line);\n\n Severity severity = consumeSeverity(line);\n\n Message message {\n file,\n lineNo.orDefault(0),\n columnRange.map([](ColumnRange c) { return c.start; }).orDefault(0),\n columnRange.map([](ColumnRange c) { return c.end; }).orDefault(0),\n kj::str(line)\n };\n\n if (severity == Severity::NOTE) {\n // Append note to previous diagnostic.\n if (addNotesToBack) {\n diagnostics.back()->notes.add(kj::mv(message));\n diagnostics.back()->message.file->markDirty();\n }\n } else {\n Diagnostic diagnostic {\n severity, kj::mv(message), {}\n };\n diagnostics.add(&file->add(kj::mv(diagnostic)));\n\n // If the notes aren't empty, then this must be a dupe diagnostic, and we don't want to\n // add duplicate notes.\n addNotesToBack = diagnostics.back()->notes.empty();\n }\n } else {\n // Doesn't appear to start with a filename. Skip.\n continue;\n }\n }\n }\n\nprivate:\n kj::Vector diagnostics;\n kj::String leftoverLog;\n bool addNotesToBack = false;\n\n void clearDiagnostics() {\n for (auto diagnostic: diagnostics) {\n diagnostic->message.file->remove(*diagnostic);\n }\n diagnostics.clear();\n addNotesToBack = false;\n }\n};\n\nclass LanguageServerImpl final: public lsp::LanguageServer::Server {\npublic:\n LanguageServerImpl(kj::Own> initializedFulfiller)\n : initializedFulfiller(kj::mv(initializedFulfiller)) {}\n KJ_DISALLOW_COPY(LanguageServerImpl);\n\n class Scope {\n public:\n Scope(LanguageServerImpl& server, SourceFileSet& files, kj::StringPtr uriPrefix)\n : server(server), files(files), uriPrefix(uriPrefix) {\n server.scope = this;\n }\n ~Scope() noexcept(false) {\n server.scope = nullptr;\n }\n\n private:\n LanguageServerImpl& server;\n SourceFileSet& files;\n kj::StringPtr uriPrefix;\n\n friend class LanguageServerImpl;\n };\n\nprotected:\n kj::Promise initialize(InitializeContext context) override {\n initializedFulfiller->fulfill();\n auto sync = context.initResults().initCapabilities().initTextDocumentSync();\n sync.setOpenClose(true);\n sync.setChange(lsp::TextDocumentSyncKind::INCREMENTAL);\n sync.setDidSave(true);\n return kj::READY_NOW;\n }\n kj::Promise shutdown(ShutdownContext context) override {\n return kj::READY_NOW;\n }\n kj::Promise exit(ExitContext context) override {\n _exit(0);\n }\n\n kj::Promise didOpen(DidOpenContext context) override {\n // Ignore.\n return kj::READY_NOW;\n }\n kj::Promise didClose(DidCloseContext context) override {\n // Ignore.\n return kj::READY_NOW;\n }\n kj::Promise didChange(DidChangeContext context) override {\n KJ_IF_MAYBE(s, scope) {\n auto params = context.getParams();\n auto uri = params.getTextDocument().getUri();\n if (uri.startsWith(s->uriPrefix)) {\n auto path = kj::decodeUriComponent(uri.slice(s->uriPrefix.size()));\n KJ_IF_MAYBE(file, s->files.findByRealPath(path)) {\n file->markStale();\n }\n }\n }\n return kj::READY_NOW;\n }\n kj::Promise didSave(DidSaveContext context) override {\n // Ignore for now.\n // TODO(someday): Start a new location map for this file and interpret future diagnostics\n // against that map rather than the current one.\n return kj::READY_NOW;\n }\n\nprivate:\n kj::Own> initializedFulfiller;\n kj::Maybe scope;\n};\n\nclass LanguageServerMain {\npublic:\n LanguageServerMain(kj::ProcessContext& context): context(context) {}\n\n kj::MainFunc getMain() {\n return kj::MainBuilder(context, \"Ekam Language Server\",\n \"Implements the VS Code Language Server Protocol to report errors from Ekam.\")\n .expectArg(\"
\", KJ_BIND_METHOD(*this, run))\n .build();\n }\n\n kj::MainBuilder::Validity run(kj::StringPtr addr) {\n auto io = kj::setupAsyncIo();\n auto fs = kj::newDiskFilesystem();\n\n auto parsedAddr = io.provider->getNetwork().parseAddress(addr).wait(io.waitScope);\n\n auto stream = kj::heap(\n io.lowLevelProvider->wrapInputFd(kj::AutoCloseFd(STDIN_FILENO)),\n io.lowLevelProvider->wrapOutputFd(kj::AutoCloseFd(STDOUT_FILENO)));\n\n auto initPaf = kj::newPromiseAndFulfiller();\n\n auto ownServer = kj::heap(kj::mv(initPaf.fulfiller));\n LanguageServerImpl& server = *ownServer;\n\n capnp::JsonRpc::ContentLengthTransport transport(*stream);\n capnp::JsonRpc jsonRpc(transport, capnp::toDynamic(kj::mv(ownServer)));\n\n auto errorTask = jsonRpc.onError()\n .then([]() {\n KJ_LOG(ERROR, \"JsonRpc.onError() resolved normally?\");\n }, [](kj::Exception&& exception) {\n KJ_LOG(ERROR, \"JSON-RPC connection failed\", exception);\n }).attach(kj::defer([]() {\n exit(1);\n })).eagerlyEvaluate(nullptr);\n\n auto client = jsonRpc.getPeer();\n\n // Wait until initialized, as vscode will ignore diagnostics delivered too soon.\n initPaf.promise.wait(io.waitScope);\n\n for (;;) {\n // Repeatedly try to connect to Ekam.\n kj::Own ekamConnection;\n for (;;) {\n KJ_IF_MAYBE(exception, kj::runCatchingExceptions([&]() {\n ekamConnection = parsedAddr->connect().wait(io.waitScope);\n })) {\n if (exception->getType() == kj::Exception::Type::DISCONNECTED) {\n io.provider->getTimer().afterDelay(2 * kj::SECONDS).wait(io.waitScope);\n } else {\n kj::throwFatalException(kj::mv(*exception));\n }\n } else {\n break;\n }\n }\n\n // Read first message from Ekam.\n kj::String homeUri;\n auto projectHome = ({\n auto message = capnp::readMessage(*ekamConnection).wait(io.waitScope);\n auto header = message->getRoot();\n auto path = kj::str(header.getProjectRoot());\n homeUri = kj::str(\"file://\", path, '/');\n KJ_ASSERT(path.startsWith(\"/\"));\n fs->getRoot().openSubdir(kj::Path::parse(path.slice(1)));\n });\n\n DirtySet dirtySet;\n SourceFileSet files(*projectHome, dirtySet);\n kj::HashMap> tasks;\n\n LanguageServerImpl::Scope serverScope(server, files, homeUri);\n kj::Promise updateLoopTask = updateLoop(dirtySet, client, homeUri);\n\n for (;;) {\n kj::Own message;\n KJ_IF_MAYBE(exception, kj::runCatchingExceptions([&]() {\n message = capnp::readMessage(*ekamConnection).wait(io.waitScope);\n })) {\n if (exception->getType() == kj::Exception::Type::DISCONNECTED) {\n // Disconnected, start over.\n break;\n } else {\n kj::throwFatalException(kj::mv(*exception));\n }\n }\n auto update = message->getRoot();\n\n if (update.getState() == proto::TaskUpdate::State::DELETED) {\n tasks.erase(update.getId());\n } else {\n auto& task = *tasks.findOrCreate(update.getId(), [&]() {\n return kj::HashMap>::Entry {\n update.getId(), kj::heap(update, files)\n };\n });\n task.update(update, files);\n }\n }\n\n // Clear all diagnostics.\n tasks.clear();\n dirtySet.shutdown();\n updateLoopTask.wait(io.waitScope);\n }\n }\n\nprivate:\n kj::ProcessContext& context;\n\n kj::Promise updateLoop(DirtySet& dirtySet,\n lsp::LanguageClient::Client client, kj::StringPtr homeUri) {\n KJ_IF_MAYBE(p, dirtySet.whenNonEmpty()) {\n return p->then([]() {\n // Delay for other messages.\n return kj::evalLast([]() {});\n }).then([&dirtySet, client, homeUri]() mutable {\n kj::Vector> promises;\n dirtySet.forEach([&](SourceFile& file) {\n auto req = client.publishDiagnosticsRequest();\n file.exportDiagnostics(homeUri, req);\n promises.add(req.send().ignoreResult());\n });\n return kj::joinPromises(promises.releaseAsArray());\n }).then([this, &dirtySet, client, homeUri]() mutable {\n return updateLoop(dirtySet, kj::mv(client), homeUri);\n });\n } else {\n return kj::READY_NOW;\n }\n }\n};\n\n} // namespace ekam\n\nKJ_MAIN(ekam::LanguageServerMain);\n" }, { "path": "src/ekam/ekam.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n\n#include \"Driver.h\"\n#include \"base/Debug.h\"\n#include \"os/DiskFile.h\"\n#include \"Action.h\"\n#include \"SimpleDashboard.h\"\n#include \"ConsoleDashboard.h\"\n#include \"CppActionFactory.h\"\n#include \"ExecPluginActionFactory.h\"\n#include \"os/OsHandle.h\"\n\nnamespace ekam {\n\nclass ExtractTypeAction : public Action {\npublic:\n ExtractTypeAction(File* file) : file(file->clone()) {}\n ~ExtractTypeAction() {}\n\n // implements Action -------------------------------------------------------------------\n bool isSilent() { return true; }\n std::string getVerb() { return \"scan\"; }\n\n Promise start(EventManager* eventManager, BuildContext* context) {\n std::vector tags;\n\n std::string name = file->canonicalName();\n\n tags.push_back(Tag::fromName(\"canonical:\" + name));\n\n while (true) {\n tags.push_back(Tag::fromFile(name));\n\n std::string::size_type slashPos = name.find_first_of('/');\n if (slashPos == std::string::npos) {\n break;\n }\n\n name.erase(0, slashPos + 1);\n }\n\n if (file->isDirectory()) {\n tags.push_back(Tag::fromName(\"directory:*\"));\n } else {\n std::string base, ext;\n splitExtension(name, &base, &ext);\n if (!ext.empty()) tags.push_back(Tag::fromName(\"filetype:\" + ext));\n }\n\n context->provide(file.get(), tags);\n\n return newFulfilledPromise();\n }\n\nprivate:\n OwnedPtr file;\n};\n\nclass ExtractTypeActionFactory : public ActionFactory {\npublic:\n ExtractTypeActionFactory() {}\n ~ExtractTypeActionFactory() {}\n\n // implements ActionFactory ------------------------------------------------------------\n void enumerateTriggerTags(std::back_insert_iterator > iter) {\n *iter++ = Tag::DEFAULT_TAG;\n }\n OwnedPtr tryMakeAction(const Tag& id, File* file) {\n return newOwned(file);\n }\n};\n\nvoid usage(const char* command, FILE* out) {\n fprintf(out,\n \"usage: %s [-hvc] [-j ] [-n []:] [-l ]\\n\"\n \"\\n\"\n \"Build code with Ekam. See https://github.io/sandstorm-io/ekam for details.\\n\"\n \"\\n\"\n \"options:\\n\"\n \" -c Run in continuous mode: when there is nothing left to build,\\n\"\n \" don't exit, but instead watch the source files for changes\\n\"\n \" and rebuild as necessary.\\n\"\n \" -j Run up to actions in parallel.\\n\"\n \" -n []: Accept network connections on the given address/port\\n\"\n \" and give real-time build status and logs to anyone who\\n\"\n \" connects. This enables e.g. `ekam-client` and various IDE\\n\"\n \" plugins.\\n\"\n \" -l Set max number of log lines to display per action. This is\\n\"\n \" kept relatively short by default because it makes the build\\n\"\n \" output noisy, but you may need to increase it if you need\\n\"\n \" to see more of a particular error log. NOTE: If you just\\n\"\n \" need a one-off, you can use `ekam-client` rather than\\n\"\n \" restarting Ekam.\\n\"\n \" -h See this help\\n\"\n \" -v Show debug logs.\\n\",\n command);\n}\n\n// =======================================================================================\n// TODO: Move file-watching code to another module.\n\nclass Watcher {\npublic:\n Watcher(OwnedPtr file, EventManager* eventManager, Driver* driver,\n bool isDirectory)\n : eventManager(eventManager), driver(driver), isDirectory(isDirectory), file(file.release()) {\n resetWatch();\n }\n\n virtual ~Watcher() {}\n\n EventManager* const eventManager;\n Driver* const driver;\n const bool isDirectory;\n OwnedPtr file;\n\n void resetWatch() {\n asyncOp.release();\n diskRef = file->getOnDisk(File::READ);\n watcher = eventManager->watchFile(diskRef->path());\n waitForEvent();\n }\n\n void waitForEvent() {\n asyncOp = eventManager->when(watcher->onChange())(\n [this](EventManager::FileChangeType changeType) {\n switch (changeType) {\n case EventManager::FileChangeType::MODIFIED:\n modified();\n break;\n case EventManager::FileChangeType::DELETED:\n deleted();\n break;\n }\n waitForEvent();\n });\n }\n\n void clearWatch() {\n asyncOp.release();\n }\n\n bool isDeleted() {\n return asyncOp == nullptr;\n }\n\n virtual void created() = 0;\n virtual void modified() = 0;\n virtual void deleted() = 0;\n virtual void reallyDeleted() = 0;\n\nprivate:\n OwnedPtr diskRef;\n OwnedPtr watcher;\n Promise asyncOp;\n};\n\nclass FileWatcher : public Watcher {\npublic:\n FileWatcher(OwnedPtr file, EventManager* eventManager, Driver* driver)\n : Watcher(file.release(), eventManager, driver, false) {}\n ~FileWatcher() {}\n\n // implements FileChangeCallback -------------------------------------------------------\n void created() {\n DEBUG_INFO << \"Source file created: \" << file->canonicalName();\n modified();\n }\n void modified() {\n DEBUG_INFO << \"Source file modified: \" << file->canonicalName();\n\n driver->addSourceFile(file.get());\n }\n void deleted() {\n if (file->isFile()) {\n // A new file was created in place of the old. Reset the watch.\n DEBUG_INFO << \"Source file replaced: \" << file->canonicalName();\n resetWatch();\n modified();\n } else {\n reallyDeleted();\n }\n }\n\n // implements Watcher ------------------------------------------------------------------\n void reallyDeleted() {\n DEBUG_INFO << \"Source file deleted: \" << file->canonicalName();\n\n clearWatch();\n driver->removeSourceFile(file.get());\n }\n};\n\nclass DirectoryWatcher : public Watcher {\n typedef OwnedPtrMap ChildMap;\npublic:\n DirectoryWatcher(OwnedPtr file, EventManager* eventManager, Driver* driver)\n : Watcher(file.release(), eventManager, driver, true) {}\n ~DirectoryWatcher() {}\n\n // implements FileChangeCallback -------------------------------------------------------\n void created() {\n driver->addSourceFile(file.get());\n modified();\n }\n void modified() {\n DEBUG_INFO << \"Directory modified: \" << file->canonicalName();\n\n OwnedPtrVector list;\n try {\n file->list(list.appender());\n } catch (OsError& e) {\n // Probably the directory has been deleted but we weren't yet notified.\n reallyDeleted();\n return;\n }\n\n ChildMap newChildren;\n\n // Build new child list, copying over child watchers where possible.\n for (int i = 0; i < list.size(); i++) {\n OwnedPtr childFile = list.release(i);\n\n OwnedPtr child;\n bool childIsDirectory = childFile->isDirectory();\n\n // When a file is deleted and replaced with a new one of the same type, we run into a lot\n // of awkward race conditions. There are three things that can happen in any order:\n // 1) Notification of file deletion.\n // 2) Notification of directory change.\n // 3) New file is created.\n //\n // Here is how we handle each possible ordering:\n // 1, 2, 3) File will not show up in directory list, so we won't transfer the watcher or\n // create a new one. It will be destroyed.\n // 1, 3, 2) child->isDeleted will be true so we'll create a new watcher to replace it.\n // 2, 1, 3) Like 1, 2, 3 except we directly call deleted() on the old child watcher from\n // this function (see below). We actually never receive the file deletion event from\n // the EventManager in this case.\n // 2, 3, 1) Same as 2, 1, 3.\n // 3, 1, 2) File watcher notices new file already exists and simply resumes watching.\n // Parent watcher thinks nothing happened.\n // 3, 2, 1) Same as 3, 1, 2.\n //\n // The last two are different if a file was replaced with a directory or vice versa:\n // 3, 1, 2) Child watcher notices replacement is a different type and so does not resume\n // watching. The parent notices child->isDeleted is true and replaces it.\n // 3, 2, 1) The parent notices that child->isDirectory does not match the type of the new\n // file, and so deletes the child watcher explicitly.\n if (!children.release(childFile.get(), &child) ||\n child->isDeleted() || child->isDirectory != childIsDirectory) {\n if (childIsDirectory) {\n child = newOwned(childFile.release(), eventManager, driver);\n } else {\n child = newOwned(childFile.release(), eventManager, driver);\n }\n child->created();\n }\n\n File* key = child->file.get(); // cannot inline due to undefined evaluation order\n newChildren.add(key, child.release());\n }\n\n // Make sure remaining children have been notified of deletion before we destroy the objects.\n for (ChildMap::Iterator iter(children); iter.next();) {\n if (!iter.value()->isDeleted()) {\n iter.value()->reallyDeleted();\n }\n }\n\n // Swap in new children.\n children.swap(&newChildren);\n }\n\n void deleted() {\n if (file->isDirectory()) {\n // A new directory was created in place of the old. Reset the watch.\n DEBUG_INFO << \"Directory replaced: \" << file->canonicalName();\n resetWatch();\n modified();\n } else {\n reallyDeleted();\n }\n }\n\n // implements Watcher ------------------------------------------------------------------\n void reallyDeleted() {\n DEBUG_INFO << \"Directory deleted: \" << file->canonicalName();\n\n clearWatch();\n driver->removeSourceFile(file.get());\n\n // Delete all children.\n for (ChildMap::Iterator iter(children); iter.next();) {\n if (!iter.value()->isDeleted()) {\n iter.value()->reallyDeleted();\n }\n }\n children.clear();\n }\n\nprivate:\n ChildMap children;\n};\n\n// =======================================================================================\n\nclass EkamLocks final: public Driver::ActivityObserver {\npublic:\n EkamLocks(File* tmp)\n : mainLock(\"tmp/.ekam-lock\",\n openLockfile(tmp->relative(\".ekam-lock\").get())),\n activeLock(\"tmp/.ekam-lock-active\",\n openLockfile(tmp->relative(\".ekam-lock-active\").get())) {}\n\n bool tryTakeMainLock() {\n while (flock(mainLock.get(), LOCK_EX | LOCK_NB) < 0) {\n if (errno == EWOULDBLOCK) {\n return false;\n } else if (errno == ENOLCK) {\n // Filesystem doesn't support locking.\n fprintf(stderr, \"WARNING: Filesystem doesn't support locking. Do not run two Ekam instances concurrently.\\n\");\n noLocking = true;\n return true;\n } else if (errno != EAGAIN) {\n throw OsError(\"flock(mainLock)\", errno);\n }\n }\n\n return true;\n }\n\n void waitForOther() {\n wrapSyscall(\"flock(activeLock)\", flock, activeLock.get(), LOCK_SH);\n\n char c = 0;\n ssize_t n = wrapSyscall(\"read(activeLock)\", read, activeLock, &c, 1);\n if (n == 1 && c == 'p') {\n failed = false;\n } else if (n == 1 && c == 'f') {\n failed = true;\n } else {\n throw std::logic_error(\"lock file contents invalid\");\n }\n\n wrapSyscall(\"flock(activeLock)\", flock, activeLock.get(), LOCK_UN);\n }\n\n bool hasFailures() {\n if (running) throw std::logic_error(\"can't check failures while still running\");\n return failed;\n }\n\n void startingAction() override {\n if (!running) {\n running = true;\n if (!noLocking) flock(activeLock.get(), LOCK_EX);\n }\n }\n\n void idle(bool hasFailures) override {\n if (running) {\n running = false;\n wrapSyscall(\"write(activeLock)\", write, activeLock.get(), hasFailures ? \"fail\" : \"pass\", 4);\n if (!noLocking) wrapSyscall(\"flock(activeLock, LOCK_UN)\", flock, activeLock.get(), LOCK_UN);\n }\n failed = hasFailures;\n }\n\nprivate:\n OsHandle mainLock;\n OsHandle activeLock;\n bool running = false;\n bool failed = false;\n bool noLocking = false;\n\n static int openLockfile(File* lockfile) {\n auto diskfile = lockfile->getOnDisk(File::UPDATE);\n return wrapSyscall(\"open(lockfile)\", open,\n diskfile->path().c_str(), O_RDWR | O_CREAT | O_CLOEXEC, 0666);\n }\n};\n\n// =======================================================================================\n\nvoid scanSourceTree(File* src, Driver* driver) {\n OwnedPtrVector fileQueue;\n\n {\n fileQueue.add(src->clone());\n }\n\n while (!fileQueue.empty()) {\n OwnedPtr current = fileQueue.releaseBack();\n\n if (current->isDirectory()) {\n OwnedPtrVector list;\n current->list(list.appender());\n for (int i = 0; i < list.size(); i++) {\n fileQueue.add(list.release(i));\n }\n }\n\n driver->addSourceFile(current.get());\n }\n}\n\nOwnedPtr getDashboard(int maxDisplayedLogLines) {\n if (!isatty(STDOUT_FILENO)) {\n return newOwned(stdout);\n }\n\n // Sanity check: make sure the window size is non-zero; otherwise something\n // is messed up about the terminal, and we should assume it doesn't work\n // correctly.\n //\n // See issue #29\n struct winsize windowSize;\n if(ioctl(STDOUT_FILENO, TIOCGWINSZ, &windowSize) < 0) {\n const char *msg = strerror(errno);\n DEBUG_WARNING\n << \"Error querying terminal size: \" << msg << \"; \"\n << \"falling back to simple output.\";\n return newOwned(stdout);\n }\n if(windowSize.ws_row == 0 || windowSize.ws_col == 0) {\n DEBUG_WARNING\n << \"Terminal size looks suspicious \"\n << \"(rows = \" << windowSize.ws_row << \", columns = \" << windowSize.ws_col << \"); \"\n << \"falling back to simple output.\";\n return newOwned(stdout);\n }\n return newOwned(stdout, maxDisplayedLogLines);\n}\n\nint main(int argc, char* argv[]) {\n signal(SIGPIPE, SIG_IGN);\n\n int maxDisplayedLogLines = 30;\n const char* command = argv[0];\n int maxConcurrentActions = 1;\n bool continuous = false;\n std::string networkDashboardAddress;\n\n while (true) {\n int opt = getopt(argc, argv, \"chvj:n:l:\");\n if (opt == -1) break;\n\n switch (opt) {\n case 'v':\n DebugMessage::setLogLevel(DebugMessage::INFO);\n break;\n case 'j': {\n char* endptr;\n maxConcurrentActions = strtoul(optarg, &endptr, 10);\n if (*endptr != '\\0') {\n fprintf(stderr, \"Expected number after -j.\\n\");\n return 1;\n }\n break;\n }\n case 'h':\n usage(command, stdout);\n return 0;\n case 'c':\n continuous = true;\n break;\n case 'n':\n networkDashboardAddress = optarg;\n break;\n case 'l': {\n char* endptr;\n maxDisplayedLogLines = strtoul(optarg, &endptr, 0);\n if (*endptr != '\\0') {\n fprintf(stderr, \"Expected number after -l.\\n\");\n return 1;\n }\n break;\n }\n default:\n usage(command, stderr);\n return 1;\n }\n }\n\n argc -= optind;\n argv += optind;\n\n if (argc > 0) {\n fprintf(stderr, \"%s: unknown argument -- %s\\n\", command, argv[0]);\n return 1;\n }\n\n DiskFile src(\"src\", NULL);\n DiskFile tmp(\"tmp\", NULL);\n DiskFile bin(\"bin\", NULL);\n DiskFile lib(\"lib\", NULL);\n DiskFile nodeModules(\"node_modules\", NULL);\n File* installDirs[BuildContext::INSTALL_LOCATION_COUNT] = { &bin, &lib, &nodeModules };\n\n if (!tmp.isDirectory()) {\n tmp.createDirectory();\n }\n\n EkamLocks locks(&tmp);\n if (!locks.tryTakeMainLock()) {\n if (continuous) {\n fprintf(stderr, \"ERROR: Ekam is already running in this directory.\\n\");\n return 1;\n } else {\n fprintf(stderr, \"Another Ekam is already running in this directory.\\n\"\n \"Waiting for build to complete...\\n\");\n locks.waitForOther();\n return locks.hasFailures() ? 1 : 0;\n }\n }\n\n OwnedPtr eventManager = newPreferredEventManager();\n\n OwnedPtr dashboard = getDashboard(maxDisplayedLogLines);\n if (!networkDashboardAddress.empty()) {\n dashboard = initNetworkDashboard(eventManager.get(), networkDashboardAddress,\n dashboard.release());\n }\n\n Driver driver(eventManager.get(), dashboard.get(), &tmp, installDirs, maxConcurrentActions,\n &locks);\n\n ExtractTypeActionFactory extractTypeActionFactcory;\n driver.addActionFactory(&extractTypeActionFactcory);\n\n CppActionFactory cppActionFactory;\n driver.addActionFactory(&cppActionFactory);\n\n ExecPluginActionFactory execPluginActionFactory;\n driver.addActionFactory(&execPluginActionFactory);\n\n OwnedPtr rootWatcher;\n if (continuous) {\n rootWatcher = newOwned(src.clone(), eventManager.get(), &driver);\n rootWatcher->modified();\n } else {\n scanSourceTree(&src, &driver);\n }\n eventManager->loop();\n\n // For debugging purposes, check for zombie processes.\n int zombieCount = 0;\n while (true) {\n int status;\n pid_t pid = wait(&status);\n if (pid < 0) {\n if (errno == ECHILD) {\n // No more children.\n break;\n } else {\n DEBUG_ERROR << \"wait: \" << strerror(errno);\n }\n } else {\n ++zombieCount;\n }\n }\n\n if (zombieCount > 0) {\n DEBUG_ERROR << \"There were \" << zombieCount\n << \" zombie processes after the event loop stopped.\";\n return 1;\n } else {\n DEBUG_INFO << \"No zombie processes detected. Hooray.\";\n return locks.hasFailures() ? 1 : 0;\n }\n}\n\n} // namespace ekam\n\nint main(int argc, char* argv[]) {\n return ekam::main(argc, argv);\n}\n" }, { "path": "src/ekam/ekam.ekam-manifest", "content": "ekam bin\nekam-client bin\nekam-langserve bin\n" }, { "path": "src/ekam/initNetworkDashboardStub.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"Dashboard.h\"\n#include \n\nnamespace ekam {\n\nOwnedPtr initNetworkDashboard(EventManager* eventManager, const std::string& address,\n OwnedPtr dashboardToWrap) {\n throw std::logic_error(\"Network dashboard support not compiled in.\");\n}\n\n} // namespace ekam\n" }, { "path": "src/ekam/langserve.capnp", "content": "@0xdf85b05eff43f858;\n# Cap'n Proto definitions for a subset of the Language Server Protocol, usable with capnp::JsonRpc.\n# Currently this is the minimum subset needed for Ekam.\n\nusing Json = import \"/capnp/compat/json.capnp\";\n\n$import \"/capnp/c++.capnp\".namespace(\"ekam::lsp\");\n\ninterface LanguageServer {\n initialize @0 InitializeParams -> (capabilities :ServerCapabilities);\n\n struct InitializeParams {\n processId @0 :UInt32; # Technically can be null.\n rootPath @1 :Text;\n rootUri @2 :DocumentUri;\n initializationOptions @3 :Json.Value;\n capabilities @4 :ClientCapabilities;\n trace @5 :Text = \"off\";\n workspaceFolders @6 :List(WorkspaceFolder);\n }\n\n shutdown @1 ();\n exit @2 () $Json.notification;\n\n didOpen @3 () $Json.notification $Json.name(\"textDocument/didOpen\");\n didClose @4 () $Json.notification $Json.name(\"textDocument/didClose\");\n # Ignoring these for now.\n\n didChange @5 (textDocument :TextDocumentIdentifier,\n # Actually VersionedTextDocumentIdentifier but we don't care.\n contentChanges :List(TextDocumentContentChangeEvent))\n $Json.notification $Json.name(\"textDocument/didChange\");\n\n didSave @6 (textDocument :TextDocumentIdentifier)\n $Json.notification $Json.name(\"textDocument/didSave\");\n # TODO(someday): Why is this never sent at present?\n}\n\ninterface LanguageClient {\n publishDiagnostics @0 (uri :DocumentUri, diagnostics :List(Diagnostic)) $Json.notification\n $Json.name(\"textDocument/publishDiagnostics\");\n}\n\nstruct ClientCapabilities {\n # TODO(someday): workspace, textDocument, experimental\n}\n\nstruct ServerCapabilities {\n textDocumentSync @0 :TextDocumentSyncOptions;\n\n # TODO(someday): more\n}\n\nstruct TextDocumentSyncOptions {\n openClose @0 :Bool = false;\n change @1 :UInt8 = TextDocumentSyncKind.none;\n didSave @2 :Bool = false;\n}\n\nstruct TextDocumentSyncKind {\n const none :UInt32 = 0;\n const full :UInt32 = 1;\n const incremental :UInt32 = 2;\n}\n\nstruct WorkspaceFolder {\n uri @0 :Text;\n name @1 :Text;\n}\n\nusing DocumentUri = Text;\n\nstruct Position {\n line @0 :UInt32;\n character @1 :UInt32;\n}\nstruct Range {\n start @0 :Position;\n end @1 :Position;\n}\nstruct Location {\n uri @0 :DocumentUri;\n range @1 :Range;\n}\n\nstruct Diagnostic {\n range @0 :Range;\n severity @1 :UInt32;\n code @2 :Text;\n source @3 :Text;\n message @4 :Text;\n relatedInformation @5 :List(RelatedInformation);\n\n struct RelatedInformation {\n location @0 :Location;\n message @1 :Text;\n }\n\n const severityError :UInt32 = 1;\n const severityWarning :UInt32 = 2;\n const severityInformation :UInt32 = 3;\n const severityHint :UInt64 = 4;\n}\n\nstruct Command {\n title @0 :Text;\n command @1 :Text;\n arguments @2 :List(Json.Value);\n}\n\nstruct TextDocumentContentChangeEvent {\n range @0 :Range;\n text @1 :Text;\n}\n\nstruct TextDocumentIdentifier {\n uri @0 :DocumentUri;\n}\n" }, { "path": "src/ekam/rules/compile.ekam-rule", "content": "#! /bin/sh\n\n# Ekam Build System\n# Author: Kenton Varda (kenton@sandstorm.io)\n# Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -eu\n\nif test $# = 0; then\n # Ekam is querying the script. Tell it that we like C++ source files.\n echo trigger filetype:.cpp\n echo trigger filetype:.cc\n echo trigger filetype:.C\n echo trigger filetype:.cxx\n echo trigger filetype:.c++\n echo trigger filetype:.c\n exit 0\nfi\n\nINPUT=$1\nshift\n\n# Set defaults. We use -O2 -DNDEBUG as default flags because the users that are most likely not\n# to specify flags are people who are just compiling someone else's code to use it, and those\n# people do not want debug builds.\nCXX=${CXX:-c++}\nCXXFLAGS=${CXXFLAGS:--O2 -DNDEBUG}\nCC=${CC:-cc}\nCFLAGS=${CFLAGS:--O2 -DNDEBUG}\n\n# Look up modifiers.\necho findModifiers compile.ekam-flags\nwhile true; do\n read MODIFIER\n if test -z \"$MODIFIER\"; then\n break\n fi\n . \"$MODIFIER\" 1>&2\ndone\n\ncase \"$INPUT\" in\n *.cpp )\n MODULE_NAME=${INPUT%.cpp}\n ;;\n *.cc )\n MODULE_NAME=${INPUT%.cc}\n ;;\n *.C )\n MODULE_NAME=${INPUT%.C}\n ;;\n *.cxx )\n MODULE_NAME=${INPUT%.cxx}\n ;;\n *.c++ )\n MODULE_NAME=${INPUT%.c++}\n ;;\n intercept.c | */intercept.c )\n # Hack: Skip interceptor. It screws everything up since it appears to define syscalls like\n # write().\n # TODO(cleanup): Do a better job detecting this.\n exit 0\n ;;\n *.c )\n MODULE_NAME=${INPUT%.c}\n CXX=${CC}\n CXXFLAGS=${CFLAGS}\n ;;\n * )\n echo \"Wrong file type: $INPUT\" >&2\n exit 1\n ;;\nesac\n\necho findProvider special:ekam-interceptor\nread INTERCEPTOR\n\nif test \"$INTERCEPTOR\" = \"\"; then\n echo \"error: couldn't find intercept.so.\" >&2\n exit 1\nfi\n\n# Ask Ekam where to put the output file. Actually, the compiler will make the same request again\n# when it runs, but we need to know the location too.\nOUTPUT=${MODULE_NAME}.o\necho newOutput \"$OUTPUT\"\nread OUTPUT_DISK_PATH\n\ncompile() {\n local TARGET_CXX=\"$1\"\n local SUFFIX=\"$2\"\n local FLAGSVAR=\"CXXFLAGS_$3\"\n shift 3\n\n local FLAGS=\"$(eval \"echo \\\"\\${$FLAGSVAR:-\\$CXXFLAGS}\\\"\")\"\n\n # Remove -Wglobal-constructors in tests because the test framework depends on registering global\n # objects, and we only care about global constructors in runtime code anyway.\n case \"$FLAGS\" in\n *-Wglobal-constructors* )\n case \"$MODULE_NAME\" in\n *-test )\n FLAGS=\"$FLAGS -Wno-global-constructors\"\n ;;\n esac\n esac\n\n # Compile! We LD_PRELOAD intercept.so to intercept open() and other filesystem calls and convert\n # them into Ekam requests. intercept.so expects file descriptors 3 and 4 to be the Ekam request\n # and response streams, so we remap them to stdout and stdin, respectively. We also remap stdout\n # itself to stderr just to make sure that if the compiler prints anything to stdout (which\n # normally it shouldn't), that does not get misinterpreted as an Ekam request.\n #\n # The DYLD_ vars are the Mac OSX equivalent of LD_PRELOAD. We don't bother checking which OS\n # we're on since the other vars will just be ignored anyway.\n LD_PRELOAD=$INTERCEPTOR DYLD_FORCE_FLAT_NAMESPACE= DYLD_INSERT_LIBRARIES=$INTERCEPTOR \\\n ${CXX_WRAPPER:-} $TARGET_CXX -I/ekam-provider/c++header $FLAGS \"$@\" -c \"/ekam-provider/canonical/$INPUT\" \\\n -o \"${MODULE_NAME}${SUFFIX}\" 3>&1 4<&0 >&2\n}\n\ncompile \"$CXX\" .o host\n\nfor TARGET in ${CROSS_TARGETS:-}; do\n SUFFIX=\"$(echo \"$TARGET\" | tr - _)\"\n case \"$(basename \"$CXX\")\" in\n *clang* )\n compile \"$CXX\" \".$TARGET.o\" \"$SUFFIX\" -target \"$TARGET\" \"-isystem/usr/$TARGET/include\"\n ;;\n * )\n compile \"$TARGET-$CXX\" \".$TARGET.o\" \"$SUFFIX\"\n ;;\n esac\ndone\n\n# TODO(someday): Generate symbols and deps separately for each target? Currently this is aimed at\n# architecture cross-compiling, not OS cross-compiling, so we expect the symbols are identical.\n# If they are not, the linker rule needs to change to understand this, too. Of course, what we\n# should really do is handle separate targets using separate build steps, but that seems to\n# require deep changes to Ekam.\n\n# Ask Ekam where to put the symbol and deps lists.\necho newOutput \"${MODULE_NAME}.o.syms\"\nread SYMFILE\necho newOutput \"${MODULE_NAME}.o.deps\"\nread DEPFILE\n\n# Generate the symbol list.\n\n# Additional flags to NM may be provided via NMFLAGS environment variables.\n# NMFLAGS is intentionally unquoted so that multiple arguments may be passed in.\n\n# The version of NM used can be supplied via the NM environment variable.\n# This can be useful, for example, if building with LTO & want to use llvm-nm.\n\n# TODO: Would be nice to use nm -C here to demangle names but it doesn't appear\n# to be supported on OSX.\n\"${NM:-nm}\" ${NMFLAGS:-} \"$OUTPUT_DISK_PATH\" > $SYMFILE\n\n# Function which reads the symbol list on stdin and writes all symbols matching\n# the given type pattern to stdout, optionally with a prefix.\nreadsyms() {\n grep '[^ ]* *['$1'] ' | sed -e 's,^[^ ]* *. \\(.*\\)$,'\"${2:-}\"'\\1,g'\n}\n\n# Construct the deps file by listing all undefined symbols.\nreadsyms U < $SYMFILE > $DEPFILE\n\n# ========================================================================================\n# Detect gtest-based tests and test support while we're here.\n# TODO(kenton): Probably should be a separate rule.\n\nIS_TEST=no\n\ncase $OUTPUT in\n */gtest_main.o )\n echo provide \"$OUTPUT_DISK_PATH\" gtest:main\n ;;\n */kj/test.o | kj/test.o )\n echo provide \"$OUTPUT_DISK_PATH\" kjtest:main\n ;;\n *_test.o | *_unittest.o | *_regtest.o | *-test.o )\n # Is this a gtest test that needs to link against gtest_main?\n if grep -q 7testing8internal23MakeAndRegisterTestInfo $DEPFILE && \\\n ! egrep -q '[^U] _?main$' $SYMFILE; then\n echo provide \"$OUTPUT_DISK_PATH\" gtest:test\n IS_TEST=yes\n fi\n\n # Is this a KJ test that needs to link against kj/test.o?\n if grep -q N2kj8TestCaseC $DEPFILE; then\n echo provide \"$OUTPUT_DISK_PATH\" kjtest:test\n IS_TEST=yes\n fi\n ;;\n * )\n # Node v0.10 exports a symbol like so:\n # NODE_MODULE_EXPORT node::node_module_struct modname ## _module = ...\n #\n # The HandleScope constructor is v8::HandleScope::HandleScope().\n if egrep -q ' D [a-z0-9]+_module' $SYMFILE && \\\n grep -q _ZN2v811HandleScopeC1Ev $DEPFILE; then\n echo provide \"$OUTPUT_DISK_PATH\" nodejs:module\n fi\n # Node v4 exports a symbol like so:\n # static node::node_module _module = ...\n #\n # Symbols may bear an \"L\" prefix to indicate constness, but not all compiler versions\n # mangle this way, so we tolerate the presence or absence of the qualifier.\n #\n # The HandleScope constructor is v8::HandleScope::HandleScope(v8::Isolate*)\n if grep -q -E ' d _ZL?7_module' $SYMFILE && \\\n grep -q _ZN2v811HandleScopeC1EPNS_7IsolateE $DEPFILE; then\n echo provide \"$OUTPUT_DISK_PATH\" nodejs:module\n fi\n ;;\nesac\n\nif [ \"$IS_TEST\" = no ]; then\n # Tell Ekam about the symbols provided by this file. But not for tests, because we don't want\n # other things to accidentally link against tests.\n readsyms ABCDGRSTV \"provide $OUTPUT_DISK_PATH c++symbol:\" < $SYMFILE\nfi\n" }, { "path": "src/ekam/rules/include.ekam-rule", "content": "#! /bin/sh\n\n# Ekam Build System\n# Author: Kenton Varda (kenton@sandstorm.io)\n# Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -eu\n\nif test $# = 0; then\n # Ekam is querying the script. Tell it that we care about headers.\n echo trigger filetype:.h\n echo trigger 'directory:*'\n echo silent\n exit 0\nfi\n\nINPUT=$1\n\nINCLUDE_NAME=$INPUT\nINCLUDE_NAME=${INCLUDE_NAME##*/src/}\nINCLUDE_NAME=${INCLUDE_NAME#src/}\nINCLUDE_NAME=${INCLUDE_NAME##*/include/}\nINCLUDE_NAME=${INCLUDE_NAME#include/}\n\necho provide \"$INPUT\" \"c++header:$INCLUDE_NAME\"\n\n# HACK: gtest likes to include things from its top-level directory.\n# TODO: Come up with a more general way for dealing with this.\nINCLUDE_NAME=${INPUT##*/gtest/}\nINCLUDE_NAME=${INCLUDE_NAME#gtest/}\nif test \"$INCLUDE_NAME\" != \"$INPUT\"; then\n echo provide \"$INPUT\" \"c++header:$INCLUDE_NAME\"\nfi\n" }, { "path": "src/ekam/rules/install.ekam-rule", "content": "#! /bin/sh\n\n# Ekam Build System\n# Author: Kenton Varda (kenton@sandstorm.io)\n# Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -eu\n\nif test $# = 0; then\n # Ekam is querying the script. Tell it that we like C++ source files.\n echo trigger filetype:.ekam-manifest\n exit 0\nfi\n\nINPUT=$1\n\necho findProvider file:$INPUT\nread INPUT_DISK_PATH\n\nexec 3<$INPUT_DISK_PATH\n\nDIRNAME=$(dirname $INPUT)\n\nwhile read FILE TARGET <&3; do\n FILE=$DIRNAME/$FILE\n echo findInput $FILE\n read FILE_DISK_PATH\n \n if test \"$FILE_DISK_PATH\" = \"\"; then\n echo \"$FILE: not found\" >&2\n exit 1\n fi\n\n case $TARGET in\n */* )\n echo install $FILE_DISK_PATH $TARGET\n ;;\n * )\n echo install $FILE_DISK_PATH $TARGET/$(basename $FILE)\n ;;\n esac\ndone\n" }, { "path": "src/ekam/rules/intercept.c", "content": "/* Ekam Build System\n * Author: Kenton Varda (kenton@sandstorm.io)\n * Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may not use this file except in compliance with the License.\n * You may obtain a copy of the License at\n *\n * http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\n\n#define _GNU_SOURCE\n#define _DARWIN_NO_64_BIT_INODE /* See stat madness, below. */\n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n\n#if __linux__\n#include \n#include \n#include \n#include \n#endif\n\nstatic const int EKAM_DEBUG = 0;\n\n#define likely(x) __builtin_expect((x),1)\n\ntypedef int open_t(const char * pathname, int flags, ...);\nvoid __attribute__((constructor)) start_interceptor() {\n if (EKAM_DEBUG) {\n static open_t* real_open;\n\n if (real_open == NULL) {\n real_open = (open_t*) dlsym(RTLD_NEXT, \"open\");\n assert(real_open != NULL);\n }\n\n int fd = real_open(\"/proc/self/cmdline\", O_RDONLY);\n char buffer[1024];\n int n = read(fd, buffer, 1024);\n close(fd);\n write(STDERR_FILENO, buffer, n);\n write(STDERR_FILENO, \"\\n\", 1);\n }\n\n#if __linux__\n /* Block the statx syscall which node 12 uses which we're unable to intercept because it doesn't\n * actually have a glibc wrapper, ugh. */\n\n#if 0 /* source of the seccomp filter below */\n ld [0] /* offsetof(struct seccomp_data, nr) */\n jeq #332, nosys /* __NR_statx */\n jmp good\n\nnosys: ret #0x00050026 /* SECCOMP_RET_ERRNO | ENOSYS */\ngood: ret #0x7fff0000 /* SECCOMP_RET_ALLOW */\n#endif\n\n static struct sock_filter filter[] = {\n { 0x20, 0, 0, 0000000000 },\n { 0x15, 1, 0, 0x0000014c },\n { 0x05, 0, 0, 0x00000001 },\n { 0x06, 0, 0, 0x00050026 },\n { 0x06, 0, 0, 0x7fff0000 },\n };\n static struct sock_fprog prog = { sizeof(filter) / sizeof(filter[0]), filter };\n\n prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0);\n syscall(__NR_seccomp, SECCOMP_SET_MODE_FILTER, 0, &prog);\n#endif\n}\n\n/****************************************************************************************/\n/* Bootstrap pthreads without linking agaist libpthread. */\n\ntypedef int pthread_once_func(pthread_once_t*, void (*)(void));\ntypedef int pthread_mutex_lock_func(pthread_mutex_t*);\ntypedef int pthread_mutex_unlock_func(pthread_mutex_t*);\n\nstatic pthread_once_func* dynamic_pthread_once = NULL;\nstatic pthread_mutex_lock_func* dynamic_pthread_mutex_lock = NULL;\nstatic pthread_mutex_unlock_func* dynamic_pthread_mutex_unlock = NULL;\n\nint fake_pthread_once(pthread_once_t* once_control, void (*init_func)(void)) {\n static pthread_mutex_t fake_once_mutex = PTHREAD_MUTEX_INITIALIZER;\n static int ONCE_INPROGRESS = 1;\n static int ONCE_DONE = 2;\n\n int initialized = __atomic_load_n(once_control, __ATOMIC_ACQUIRE);\n if (likely(initialized & ONCE_DONE)) return 0;\n\n // TODO(soon): Remove the dynamic_pthread_mutex usage once dynamic_pthread_once isn't forced to\n // unconditionally use fake_pthread_once. See init_pthreads.\n\n // The real implementation is typically more complex to try to make sure that concurrent\n // initialization of different pthread_once doesn't have a false dependency on a global mutex.\n // However, we only have a single such usage here & that would be overkill to try to implement on\n // MacOS and Linux. Use a single mutex anyway because it would have largely the same effect. It\n // also has additional complexities to handle things like forking and that too isn't a concern\n // here.\n dynamic_pthread_mutex_lock(&fake_once_mutex);\n pthread_once_t expected = PTHREAD_ONCE_INIT;\n bool updated =__atomic_compare_exchange_n(once_control, &expected, ONCE_INPROGRESS, false,\n __ATOMIC_RELEASE, __ATOMIC_RELAXED);\n if (likely(!updated)) {\n assert(__atomic_load_n(&expected, __ATOMIC_RELAXED) == ONCE_DONE);\n dynamic_pthread_mutex_unlock(&fake_once_mutex);\n return 0;\n }\n init_func();\n __atomic_store_n(once_control, ONCE_DONE, __ATOMIC_RELEASE);\n dynamic_pthread_mutex_unlock(&fake_once_mutex);\n return 0;\n}\nint fake_pthread_mutex_lock(pthread_mutex_t* mutex) { return 0; }\nint fake_pthread_mutex_unlock(pthread_mutex_t* mutex) { return 0; }\n\nvoid init_pthreads() {\n if (dynamic_pthread_once == NULL) {\n dynamic_pthread_once =\n (pthread_once_func*) dlsym(RTLD_DEFAULT, \"pthread_once\");\n }\n /* TODO: For some reason the pthread_once returned by dlsym() doesn't do anything? */\n // Update 7/13/2021 - on Arch Linux pthread_once seems to work fine. Not tested on Buster so\n // unclear if this was a glibc bug that had been fixed at some point or there's something subtle\n // about when this breaks. There's an assert added in init_streams that ensures the functor\n // invoked.\n if (dynamic_pthread_once == NULL || 1) {\n dynamic_pthread_once = &fake_pthread_once;\n }\n\n if (dynamic_pthread_mutex_lock == NULL) {\n dynamic_pthread_mutex_lock =\n (pthread_mutex_lock_func*) dlsym(RTLD_DEFAULT, \"pthread_mutex_lock\");\n }\n if (dynamic_pthread_mutex_lock == NULL) {\n dynamic_pthread_mutex_lock = &fake_pthread_mutex_lock;\n }\n\n if (dynamic_pthread_mutex_unlock == NULL) {\n dynamic_pthread_mutex_unlock =\n (pthread_mutex_unlock_func*) dlsym(RTLD_DEFAULT, \"pthread_mutex_unlock\");\n }\n if (dynamic_pthread_mutex_lock == NULL) {\n dynamic_pthread_mutex_lock = &fake_pthread_mutex_unlock;\n }\n}\n\n/****************************************************************************************/\n\n#define EKAM_CALL_FILENO 3\n#define EKAM_RETURN_FILENO 4\nstatic FILE* ekam_call_stream;\nstatic FILE* ekam_return_stream;\n\nstatic char current_dir[PATH_MAX + 1];\n\nstatic pthread_once_t init_once_control = PTHREAD_ONCE_INIT;\n\ntypedef ssize_t readlink_t(const char *path, char *buf, size_t bufsiz);\nstatic readlink_t* real_readlink = NULL;\n\nstatic bool bypass_interceptor = false;\n\nstatic void init_bypass_interceptor() {\n static const char* sanitizer_env_vars[] = {\n \"ASAN_SYMBOLIZER_PATH\",\n \"MSAN_SYMBOLIZER_PATH\",\n \"LLVM_SYMBOLIZER_PATH\",\n NULL,\n };\n\n char real_exe[PATH_MAX + 1];\n bool real_exe_initialized = false;\n\n assert(real_readlink != NULL);\n\n for (size_t i = 0; sanitizer_env_vars[i]; i++) {\n const char* sanitizer_path = getenv(sanitizer_env_vars[i]);\n if (sanitizer_path != NULL) {\n if (!real_exe_initialized) {\n ssize_t result = real_readlink(\"/proc/self/exe\", real_exe, PATH_MAX);\n if (result == -1) {\n fprintf(stderr, \"Failed to readlink(/proc/self/exe): %s (%d)\\n\", strerror(errno), errno);\n abort();\n }\n\n real_exe_initialized = true;\n }\n\n if (strcmp(real_exe, sanitizer_path) == 0) {\n bypass_interceptor = true;\n break;\n }\n }\n }\n}\n\nstatic void init_streams_once() {\n static bool initialized = false;\n if (__atomic_exchange_n(&initialized, true, __ATOMIC_RELEASE)) {\n fprintf(stderr, \"pthread_once is broken\\n\");\n abort();\n }\n\n assert(ekam_call_stream == NULL);\n assert(ekam_return_stream == NULL);\n\n assert(real_readlink == NULL);\n real_readlink = (readlink_t*) dlsym(RTLD_NEXT, \"readlink\");\n assert(real_readlink != NULL);\n\n init_bypass_interceptor();\n\n if (bypass_interceptor) {\n /* Bypass any further initialization as it will fail. */\n return;\n }\n\n ekam_call_stream = fdopen(EKAM_CALL_FILENO, \"w\");\n if (ekam_call_stream == NULL) {\n fprintf(stderr, \"fdopen(EKAM_CALL_FILENO): error %d\\n\", errno);\n abort();\n }\n ekam_return_stream = fdopen(EKAM_RETURN_FILENO, \"r\");\n if (ekam_return_stream == NULL) {\n fprintf(stderr, \"fdopen(EKAM_CALL_FILENO): error %d\\n\", errno);\n abort();\n }\n if (getcwd(current_dir, PATH_MAX) == NULL) {\n fprintf(stderr, \"getcwd(): error %d\\n\", errno);\n abort();\n }\n strcat(current_dir, \"/\");\n\n}\n\nstatic void init_streams() {\n init_pthreads();\n dynamic_pthread_once(&init_once_control, &init_streams_once);\n // This assert makes sure the pthread_once call actually invokes the initializer.\n assert(bypass_interceptor || (ekam_call_stream != NULL && ekam_return_stream != NULL));\n}\n\n/****************************************************************************************/\n\ntypedef enum usage {\n READ,\n WRITE\n} usage_t;\n\nstatic const char TAG_PROVIDER_PREFIX[] = \"/ekam-provider/\";\nstatic const char TMP[] = \"/tmp\";\nstatic const char VAR_TMP[] = \"/var/tmp\";\nstatic const char TMP_PREFIX[] = \"/tmp/\";\nstatic const char VAR_TMP_PREFIX[] = \"/var/tmp/\";\nstatic const char PROC_PREFIX[] = \"/proc/\";\n\nstatic usage_t last_usage = READ;\nstatic char last_path[PATH_MAX] = \"\";\nstatic char last_result[PATH_MAX] = \"\";\npthread_mutex_t cache_mutex = PTHREAD_MUTEX_INITIALIZER;\n\nstatic void cache_result(const char* input, const char* output, usage_t usage) {\n dynamic_pthread_mutex_lock(&cache_mutex);\n last_usage = usage;\n strcpy(last_path, input);\n strcpy(last_result, output);\n dynamic_pthread_mutex_unlock(&cache_mutex);\n}\n\nstatic int get_cached_result(const char* pathname, char* buffer, usage_t usage) {\n int result = 0;\n dynamic_pthread_mutex_lock(&cache_mutex);\n if (usage == last_usage && strcmp(pathname, last_path) == 0) {\n strcpy(buffer, last_result);\n result = 1;\n }\n dynamic_pthread_mutex_unlock(&cache_mutex);\n return result;\n}\n\nstatic void canonicalizePath(char* path) {\n /* Preconditions:\n * - path has already been determined to be relative, perhaps because the pointer actually points\n * into the middle of some larger path string, in which case it must point to the character\n * immediately after a '/'.\n */\n\n /* Invariants:\n * - src points to the beginning of a path component.\n * - dst points to the location where the path component should end up, if it is not special.\n * - src == path or src[-1] == '/'.\n * - dst == path or dst[-1] == '/'.\n */\n\n char* src = path;\n char* dst = path;\n char* locked = dst; /* dst cannot backtrack past this */\n char* partEnd;\n int hasMore;\n\n for (;;) {\n while (*src == '/') {\n /* Skip duplicate slash. */\n ++src;\n }\n\n partEnd = strchr(src, '/');\n hasMore = partEnd != NULL;\n if (hasMore) {\n *partEnd = '\\0';\n } else {\n partEnd = src + strlen(src);\n }\n\n if (strcmp(src, \".\") == 0) {\n /* Skip it. */\n } else if (strcmp(src, \"..\") == 0) {\n if (dst > locked) {\n /* Backtrack over last path component. */\n --dst;\n while (dst > locked && dst[-1] != '/') --dst;\n } else {\n locked += 3;\n goto copy;\n }\n } else {\n /* Copy if needed. */\n copy:\n if (dst < src) {\n memmove(dst, src, partEnd - src);\n dst += partEnd - src;\n } else {\n dst = partEnd;\n }\n *dst++ = '/';\n }\n\n if (hasMore) {\n src = partEnd + 1;\n } else {\n /* Oops, we have to remove the trailing '/'. */\n if (dst == path) {\n /* Oops, there is no trailing '/'. We have to return \".\". */\n strcpy(path, \".\");\n } else {\n /* Remove the trailing '/'. Note that this means that opening the file will work even\n * if it is not a directory, where normally it should fail on non-directories when a\n * trailing '/' is present. If this is a problem, we need to add some sort of special\n * handling for this case where we stat() it separately to check if it is a directory,\n * because Ekam findInput will not accept a trailing '/'.\n */\n --dst;\n *dst = '\\0';\n }\n return;\n }\n }\n}\n\nstatic bool path_has_prefix(const char* pathname, const char* prefix, size_t prefix_length) {\n // Returns true if pathname is a child within prefix or if it matches prefix directly. Prefix must\n // always end with a trailing '/'.\n assert(prefix_length > 1);\n assert(prefix_length <= PATH_MAX);\n assert(prefix[prefix_length - 1] == '/');\n if (strncmp(pathname, prefix, prefix_length - 1) == 0) {\n return pathname[prefix_length - 1] == '\\0' || pathname[prefix_length - 1] == '/';\n }\n\n return false;\n}\n\nstatic bool bypass_remap(const char *pathname) {\n int debug = EKAM_DEBUG;\n\n if (pathname[0] != '/') {\n // Only absolute paths are allowed to bypass the remap.\n return false;\n }\n\n const char* bypass_dirs = getenv(\"EKAM_REMAP_BYPASS_DIRS\");\n if (bypass_dirs == NULL) {\n return false;\n }\n\n bool found = false;\n\n while (bypass_dirs && *bypass_dirs) {\n char* separator_location = strchr(bypass_dirs, ':');\n size_t bypass_dir_length;\n if (separator_location) {\n bypass_dir_length = separator_location - bypass_dirs;\n } else {\n bypass_dir_length = strlen(bypass_dirs);\n }\n\n if (bypass_dirs[0] != '/') {\n fprintf(stderr, \"Bypass directory '%s' isn't an absolute path. Skipping.\\n\", bypass_dirs);\n } else if (bypass_dirs[bypass_dir_length - 1] != '/') {\n fprintf(stderr, \"Bypass directory '%s' doesn't end in trailing '/'. Skipping.\\n\", bypass_dirs);\n } else if (path_has_prefix(pathname, bypass_dirs, bypass_dir_length)) {\n if (debug) {\n fprintf(stderr, \"Bypass found for %s\\n\", pathname);\n }\n found = true;\n break;\n }\n\n if (separator_location) {\n bypass_dirs = separator_location + 1;\n } else {\n bypass_dirs = NULL;\n }\n }\n\n return found;\n}\n\nstatic bool is_temporary_dir(const char *pathname) {\n size_t cwd_len = strlen(current_dir) - 1;\n if (strncmp(pathname, current_dir, cwd_len) == 0\n && (pathname[cwd_len] == '/' || pathname[cwd_len] == '\\0')) {\n // Somewhere under our working directory. If our working directory happens\n // to be inside one of the temporary dirs, then we need to be careful to\n // *not* treat this as a temporary file, otherwise this will apply to basicaly\n // everything, and strange things will happen.\n return false;\n }\n\n // TODO(soon): Simplify the logic to use `path_has_prefix`.\n if (strcmp(pathname, TMP) == 0 ||\n strcmp(pathname, VAR_TMP) == 0 ||\n strncmp(pathname, TMP_PREFIX, strlen(TMP_PREFIX)) == 0 ||\n strncmp(pathname, VAR_TMP_PREFIX, strlen(VAR_TMP_PREFIX)) == 0 ||\n strncmp(pathname, PROC_PREFIX, strlen(PROC_PREFIX)) == 0) {\n return true;\n }\n\n#if defined(__linux__)\n#define SYSTEMD_TMPDIR_PREFIX \"/run/user/\"\n if (path_has_prefix(pathname, SYSTEMD_TMPDIR_PREFIX, strlen(SYSTEMD_TMPDIR_PREFIX))) {\n return true;\n }\n#else\n (void) pathname;\n#endif\n\n return false;\n}\n\nstatic const char* remap_file(const char* syscall_name, const char* pathname,\n char* buffer, usage_t usage) {\n char* pos;\n int debug = EKAM_DEBUG;\n\n /* Ad-hoc debugging can be accomplished by setting debug = 1 when a particular file pattern\n * is matched. */\n\n if (debug) {\n fprintf(stderr, \"remap for %s (%s): %s\\n\",\n syscall_name, (usage == READ ? \"read\" : \"write\"), pathname);\n }\n\n init_streams();\n\n if (bypass_interceptor) {\n if (debug) fprintf(stderr, \"Bypassing interceptor for %s on %s\\n\", syscall_name, pathname);\n return pathname;\n }\n\n if (strlen(pathname) >= PATH_MAX) {\n /* Too long. */\n if (debug) fprintf(stderr, \" name too long\\n\");\n errno = ENAMETOOLONG;\n return NULL;\n }\n\n if (get_cached_result(pathname, buffer, usage)) {\n if (debug) fprintf(stderr, \" cached: %s\\n\", buffer);\n return buffer;\n }\n\n flockfile(ekam_call_stream);\n\n if (strncmp(pathname, TAG_PROVIDER_PREFIX, strlen(TAG_PROVIDER_PREFIX)) == 0) {\n /* A tag reference. Construct the tag name in |buffer|. */\n strcpy(buffer, pathname + strlen(TAG_PROVIDER_PREFIX));\n\n if (usage == READ) {\n /* Change first slash to a colon to form a tag. E.g. \"header/foo.h\" becomes\n * \"header:foo.h\". */\n pos = strchr(buffer, '/');\n if (pos == NULL) {\n /* This appears to be a tag type without a name, so it should look like a directory.\n * We can use the current directory. TODO: Return some fake empty directory instead. */\n funlockfile(ekam_call_stream);\n strcpy(buffer, \".\");\n if (debug) fprintf(stderr, \" is directory\\n\");\n return buffer;\n }\n *pos = ':';\n canonicalizePath(pos + 1);\n\n if (strcmp(buffer, \"canonical:.\") == 0) {\n /* HACK: Don't try to remap top directory. */\n funlockfile(ekam_call_stream);\n if (debug) fprintf(stderr, \" current directory\\n\");\n return \"src\";\n }\n }\n\n /* Ask ekam to remap the file name. */\n fputs(usage == READ ? \"findProvider \" : \"newProvider \", ekam_call_stream);\n fputs(buffer, ekam_call_stream);\n fputs(\"\\n\", ekam_call_stream);\n } else if (is_temporary_dir(pathname)) {\n /* Temp file or /proc. Ignore. */\n funlockfile(ekam_call_stream);\n if (debug) fprintf(stderr, \" temp file: %s\\n\", pathname);\n return pathname;\n } else if (bypass_remap(pathname)) {\n funlockfile(ekam_call_stream);\n if (debug) fprintf(stderr, \" bypassed file: %s\\n\", pathname);\n return pathname;\n } else {\n if (strncmp(pathname, current_dir, strlen(current_dir)) == 0 &&\n strncmp(pathname + strlen(current_dir), \"deps/\", 5) != 0) {\n /* The app is trying to open files in the current directory by absolute path. Treat it\n * exactly as if it had used a relative path. We make a special exception for the directory\n * `deps`, to allow e.g. executing binaries found there without mapping them into the\n * common source tree. */\n pathname = pathname + strlen(current_dir);\n } else if (pathname[0] == '/' ||\n strncmp(pathname, \"deps/\", 5) == 0) {\n /* Absolute path or under `deps`. Note the access but don't remap. */\n if (usage == WRITE) {\n /* Cannot write to absolute paths. */\n funlockfile(ekam_call_stream);\n errno = EACCES;\n if (debug) fprintf(stderr, \" absolute path, can't write\\n\");\n return NULL;\n }\n\n fputs(\"noteInput \", ekam_call_stream);\n fputs(pathname, ekam_call_stream);\n fputs(\"\\n\", ekam_call_stream);\n fflush(ekam_call_stream);\n if (ferror_unlocked(ekam_call_stream)) {\n funlockfile(ekam_call_stream);\n fprintf(stderr, \"error: Ekam call stream broken.\\n\");\n abort();\n }\n cache_result(pathname, pathname, usage);\n funlockfile(ekam_call_stream);\n if (debug) fprintf(stderr, \" absolute path: %s\\n\", pathname);\n return pathname;\n }\n\n /* Path in current directory. */\n strcpy(buffer, pathname);\n canonicalizePath(buffer);\n if (strcmp(buffer, \".\") == 0) {\n /* HACK: Don't try to remap current directory. */\n funlockfile(ekam_call_stream);\n if (debug) fprintf(stderr, \" current directory\\n\");\n return \".\";\n } else {\n /* Ask ekam to remap the file name. */\n fputs(usage == READ ? \"findInput \" : \"newOutput \", ekam_call_stream);\n fputs(buffer, ekam_call_stream);\n fputs(\"\\n\", ekam_call_stream);\n }\n }\n\n fflush(ekam_call_stream);\n if (ferror_unlocked(ekam_call_stream)) {\n funlockfile(ekam_call_stream);\n fprintf(stderr, \"error: Ekam call stream broken.\\n\");\n abort();\n }\n\n /* Carefully lock the return stream then unlock the call stream, so that we know that\n * responses will be received in the correct order. */\n flockfile(ekam_return_stream);\n funlockfile(ekam_call_stream);\n\n /* Read response from Ekam. */\n if (fgets(buffer, PATH_MAX, ekam_return_stream) == NULL) {\n funlockfile(ekam_return_stream);\n fprintf(stderr, \"error: Ekam return stream broken.\\n\");\n abort();\n }\n\n /* Done reading. */\n funlockfile(ekam_return_stream);\n\n /* Remove the trailing newline. */\n pos = strchr(buffer, '\\n');\n if (pos == NULL) {\n fprintf(stderr, \"error: Path returned from Ekam was too long.\\n\");\n abort();\n }\n *pos = '\\0';\n\n if (*buffer == '\\0') {\n /* Not found. */\n errno = ENOENT;\n if (debug) fprintf(stderr, \" ekam says no such file\\n\");\n return NULL;\n }\n\n cache_result(pathname, buffer, usage);\n\n if (debug) fprintf(stderr, \" remapped to: %s\\n\", buffer);\n return buffer;\n}\n\n/****************************************************************************************/\n\n#define WRAP(RETURNTYPE, NAME, PARAMTYPES, PARAMS, USAGE, ERROR_RESULT) \\\n typedef RETURNTYPE NAME##_t PARAMTYPES; \\\n RETURNTYPE NAME PARAMTYPES { \\\n static NAME##_t* real_##NAME = NULL; \\\n char buffer[PATH_MAX]; \\\n \\\n if (real_##NAME == NULL) { \\\n real_##NAME = (NAME##_t*) dlsym(RTLD_NEXT, #NAME); \\\n assert(real_##NAME != NULL); \\\n } \\\n \\\n path = remap_file(#NAME, path, buffer, USAGE); \\\n if (path == NULL) return ERROR_RESULT; \\\n return real_##NAME PARAMS; \\\n } \\\n RETURNTYPE _##NAME PARAMTYPES { \\\n return NAME PARAMS; \\\n }\n\nWRAP(int, chdir, (const char* path), (path), READ, -1)\nWRAP(int, chmod, (const char* path, mode_t mode), (path, mode), WRITE, -1)\nWRAP(int, lchmod, (const char* path, mode_t mode), (path, mode), WRITE, -1)\nWRAP(int, unlink, (const char* path), (path), WRITE, -1)\nWRAP(int, link, (const char* target, const char* path), (target, path), WRITE, -1)\nWRAP(int, symlink, (const char* target, const char* path), (target, path), WRITE, -1)\nWRAP(int, execve, (const char* path, char* const argv[], char* const envp[]),\n (path, argv, envp), READ, -1);\n\n#ifdef __APPLE__\nWRAP(int, stat, (const char* path, struct stat* sb), (path, sb), READ, -1)\nWRAP(int, lstat, (const char* path, struct stat* sb), (path, sb), READ, -1)\n\n/* OSX defines an alternate version of stat with 64-bit inodes. */\nWRAP(int, stat64, (const char* path, struct stat64* sb), (path, sb), READ, -1)\n\n/* In some crazy attempt to transition the regular \"stat\" call to use 64-bit inodes, Apple\n * resorted to some sort of linker magic in which calls to stat() in newly-compiled code\n * actually go to _stat$INODE64(), which appears to be identical to stat64(). We disabled\n * this by defining _DARWIN_NO_64_BIT_INODE, above, but we need to intercept all versions\n * of stat, including the $INODE64 version. Let's avoid any dependency on stat64, though,\n * since it is \"deprecated\". So, we just make the stat buf pointer opaque. */\ntypedef int stat_inode64_t(const char* path, void* sb);\nint stat_inode64(const char* path, void* sb) __asm(\"_stat$INODE64\");\nint stat_inode64(const char* path, void* sb) {\n static stat_inode64_t* real_stat_inode64 = NULL;\n char buffer[PATH_MAX];\n\n if (real_stat_inode64 == NULL) {\n real_stat_inode64 = (stat_inode64_t*) dlsym(RTLD_NEXT, \"stat$INODE64\");\n assert(real_stat_inode64 != NULL);\n }\n\n path = remap_file(\"_stat$INODE64\", path, buffer, READ);\n if (path == NULL) return -1;\n return real_stat_inode64(path, sb);\n}\n\n/* Cannot intercept intra-libc function calls on OSX, so we must intercept fopen() as well. */\nWRAP(FILE*, fopen, (const char* path, const char* mode), (path, mode),\n mode[0] == 'w' || mode[0] == 'a' ? WRITE : READ, NULL)\nWRAP(FILE*, freopen, (const char* path, const char* mode, FILE* file), (path, mode, file),\n mode[0] == 'w' || mode[0] == 'a' ? WRITE : READ, NULL)\n\n/* And remove(). */\nWRAP(int, remove, (const char* path), (path), WRITE, -1)\n\n/* Called by access(), below. */\nstatic int direct_stat(const char* path, struct stat* sb) {\n static stat_t* real_stat = NULL;\n\n if (real_stat == NULL) {\n real_stat = (stat_t*) dlsym(RTLD_NEXT, \"stat\");\n assert(real_stat != NULL);\n }\n\n return real_stat(path, sb);\n}\n\n#elif defined(__linux__)\n\nWRAP(int, __xstat, (int ver, const char* path, struct stat* sb), (ver, path, sb), READ, -1)\n//WRAP(int, __lxstat, (int ver, const char* path, struct stat* sb), (ver, path, sb), READ, -1)\nWRAP(int, __xstat64, (int ver, const char* path, struct stat64* sb), (ver, path, sb), READ, -1)\n//WRAP(int, __lxstat64, (int ver, const char* path, struct stat64* sb), (ver, path, sb), READ, -1)\n\n/* Within the source tree, we make all symbolic links look like hard links. Otherwise, if a\n * symlink in the source tree pointed outside of it, and if a tool decided to read back that link\n * and follow it manually, we'd no longer know that it was accessing something that was meant to\n * be treated as source.\n *\n * To implement this, we redirect lstat -> stat when the file is in the current directory. */\ntypedef int __lxstat_t (int ver, const char* path, struct stat* sb);\nint __lxstat (int ver, const char* path, struct stat* sb) {\n static __lxstat_t* real___lxstat = NULL;\n static __xstat_t* real___xstat = NULL;\n char buffer[PATH_MAX];\n\n if (real___lxstat == NULL) {\n real___lxstat = (__lxstat_t*) dlsym(RTLD_NEXT, \"__lxstat\");\n assert(real___lxstat != NULL);\n }\n if (real___xstat == NULL) {\n real___xstat = (__lxstat_t*) dlsym(RTLD_NEXT, \"__xstat\");\n assert(real___xstat != NULL);\n }\n\n path = remap_file(\"__lxstat\", path, buffer, READ);\n if (path == NULL) return -1;\n if (path[0] == '/') {\n return real___lxstat(ver, path, sb);\n } else {\n return real___xstat(ver, path, sb);\n }\n}\nint ___lxstat (int ver, const char* path, struct stat* sb) {\n return __lxstat (ver, path, sb);\n}\n\ntypedef int __lxstat64_t (int ver, const char* path, struct stat64* sb);\nint __lxstat64 (int ver, const char* path, struct stat64* sb) {\n static __lxstat64_t* real___lxstat64 = NULL;\n static __xstat64_t* real___xstat64 = NULL;\n char buffer[PATH_MAX];\n\n if (real___lxstat64 == NULL) {\n real___lxstat64 = (__lxstat64_t*) dlsym(RTLD_NEXT, \"__lxstat64\");\n assert(real___lxstat64 != NULL);\n }\n if (real___xstat64 == NULL) {\n real___xstat64 = (__lxstat64_t*) dlsym(RTLD_NEXT, \"__xstat64\");\n assert(real___xstat64 != NULL);\n }\n\n path = remap_file(\"__lxstat64\", path, buffer, READ);\n if (path == NULL) return -1;\n if (path[0] == '/') {\n return real___lxstat64(ver, path, sb);\n } else {\n return real___xstat64(ver, path, sb);\n }\n}\nint ___lxstat64 (int ver, const char* path, struct stat64* sb) {\n return __lxstat64 (ver, path, sb);\n}\n\n#ifndef _STAT_VER\n/* glibc 2.33+ no longer defines `stat()` as an inline wrapper in the header, but instead exports\n * it as a regular symbol. So, we need to intercept it. And, we need to give `lstat()` special\n * treatment just like `__lxstat()` above. */\n\nWRAP(int, stat, (const char* path, struct stat* sb), (path, sb), READ, -1)\nWRAP(int, stat64, (const char* path, struct stat64* sb), (path, sb), READ, -1)\n//WRAP(int, lstat, (const char* path, struct stat* sb), (path, sb), READ, -1)\n\ntypedef int lstat_t (const char* path, struct stat* sb);\nint lstat(const char* path, struct stat* sb) {\n static lstat_t* real_lstat = NULL;\n static stat_t* real_stat = NULL;\n char buffer[PATH_MAX];\n\n if (real_lstat == NULL) {\n real_lstat = (lstat_t*) dlsym(RTLD_NEXT, \"lstat\");\n assert(real_lstat != NULL);\n }\n if (real_stat == NULL) {\n real_stat = (lstat_t*) dlsym(RTLD_NEXT, \"stat\");\n assert(real_stat != NULL);\n }\n\n path = remap_file(\"lstat\", path, buffer, READ);\n if (path == NULL) return -1;\n if (path[0] == '/') {\n return real_lstat(path, sb);\n } else {\n return real_stat(path, sb);\n }\n}\nint _lstat(const char* path, struct stat* sb) {\n return lstat(path, sb);\n}\n\n#endif /* defined(_STAT_VER) */\n\n/* Cannot intercept intra-libc function calls on Linux, so we must intercept fopen() as well. */\nWRAP(FILE*, fopen, (const char* path, const char* mode), (path, mode),\n mode[0] == 'w' || mode[0] == 'a' ? WRITE : READ, NULL)\nWRAP(FILE*, fopen64, (const char* path, const char* mode), (path, mode),\n mode[0] == 'w' || mode[0] == 'a' ? WRITE : READ, NULL)\nWRAP(FILE*, freopen, (const char* path, const char* mode, FILE* file), (path, mode, file),\n mode[0] == 'w' || mode[0] == 'a' ? WRITE : READ, NULL)\nWRAP(FILE*, freopen64, (const char* path, const char* mode, FILE* file), (path, mode, file),\n mode[0] == 'w' || mode[0] == 'a' ? WRITE : READ, NULL)\n\n/* And remove(). */\nWRAP(int, remove, (const char* path), (path), WRITE, -1)\n\n/* And dlopen(). */\nWRAP(void*, dlopen, (const char* path, int flag), (path, flag), READ, NULL)\n\n/* And exec*(). The PATH-checking versions need special handling. */\nWRAP(int, execv, (const char* path, char* const argv[]), (path, argv), READ, -1)\n\ntypedef int execvpe_t(const char* path, char* const argv[], char* const envp[]);\nint execvpe(const char* path, char* const argv[], char* const envp[]) {\n static execvpe_t* real_execvpe = NULL;\n char buffer[PATH_MAX];\n\n if (real_execvpe == NULL) {\n real_execvpe = (execvpe_t*) dlsym(RTLD_NEXT, \"execvpe\");\n assert(real_execvpe != NULL);\n }\n\n /* If the path does not contain a '/', PATH resolution will be done, so we don't want to\n * remap.\n */\n if (strchr(path, '/') != NULL) {\n path = remap_file(\"execvpe\", path, buffer, READ);\n if (path == NULL) return -1;\n }\n return real_execvpe(path, argv, envp);\n}\nint _execvpe (const char* path, char* const argv[], char* const envp[]) {\n return execvpe(path, argv, envp);\n}\n\ntypedef int execvp_t(const char* path, char* const argv[]);\nint execvp(const char* path, char* const argv[]) {\n static execvp_t* real_execvp = NULL;\n char buffer[PATH_MAX];\n\n if (real_execvp == NULL) {\n real_execvp = (execvp_t*) dlsym(RTLD_NEXT, \"execvp\");\n assert(real_execvp != NULL);\n }\n\n /* If the path does not contain a '/', PATH resolution will be done, so we don't want to\n * remap.\n */\n if (strchr(path, '/') != NULL) {\n path = remap_file(\"execvp\", path, buffer, READ);\n if (path == NULL) return -1;\n }\n return real_execvp(path, argv);\n}\nint _execvp (const char* path, char* const argv[]) {\n return execvp(path, argv);\n}\n\n/* Called by access(), below. */\nstatic int direct_stat(const char* path, struct stat* sb) {\n#ifdef _STAT_VER\n // glibc <2.33 defines `stat()` as an inline wrapper in the header file, which calls __xstat().\n static __xstat_t* real_xstat = NULL;\n\n if (real_xstat == NULL) {\n real_xstat = (__xstat_t*) dlsym(RTLD_NEXT, \"__xstat\");\n assert(real_xstat != NULL);\n }\n\n return real_xstat(_STAT_VER, path, sb);\n#else /* defined(_STAT_VER) */\n // glibc 2.33+ no longer defines stat() as an inline wrapper in the header, but instead exports\n // it as a regular symbol.\n static stat_t* real_stat = NULL;\n\n if (real_stat == NULL) {\n real_stat = (stat_t*) dlsym(RTLD_NEXT, \"stat\");\n assert(real_stat != NULL);\n }\n\n return real_stat(path, sb);\n#endif /* defined(_STAT_VER), #else */\n}\n\nssize_t readlink(const char *path, char *buf, size_t bufsiz) {\n char buffer[PATH_MAX];\n\n path = remap_file(\"readlink\", path, buffer, READ);\n if (path == NULL) return -1;\n if (path[0] != '/') return EINVAL; // no links in source directory\n return real_readlink(path, buf, bufsiz);\n}\n\n#else\n\nWRAP(int, stat, (const char* path, struct stat* sb), (path, sb), READ, -1)\nWRAP(int, lstat, (const char* path, struct stat* sb), (path, sb), READ, -1)\n\n/* Called by access(), below. */\nstatic int direct_stat(const char* path, struct stat* sb) {\n static stat_t* real_stat = NULL;\n\n if (real_stat == NULL) {\n real_stat = (stat_t*) dlsym(RTLD_NEXT, \"stat\");\n assert(real_stat != NULL);\n }\n\n return real_stat(path, sb);\n}\n\n#endif /* platform */\n\nstatic int intercepted_open(const char * pathname, int flags, va_list args) {\n static open_t* real_open;\n char buffer[PATH_MAX];\n const char* remapped;\n\n#if __linux__\n // The new TCMalloc will attempt to open various files in /sys/ to read information about the CPU\n // (this is implemented within Abseil). If we actually do the dlsym below, we'll end up causing\n // a deadlock within TCMalloc because `dlsym` will cause TCMalloc's constructor to try to\n // initialize again, but the open call is being done within that. I thought perhaps this could be\n // resolved upstream (https://github.com/google/tcmalloc/issues/78) but after looking into it\n // more, there's a fundamental issue with calls to open these files causing trying to read these\n // files again. Since Abseil has \"call_once\" semantics to read the files in /sys/, even if you\n // tried to pre-cache it in TCMalloc, Abseil would end up deadlocking when you reentrantly tried\n // to initialize the CPU frequency.\n // Technically this only needs to be done if `real_open` isn't resolved, but my thought was that\n // there's no real use-case where Ekam would want to intercept /sys/ & thus this simplifies me\n // having to do more thorough testing.\n if (strncmp(\"/sys/\", pathname, strlen(\"/sys/\")) == 0) {\n mode_t mode = 0;\n if (flags & O_CREAT) {\n mode = va_arg(args, int);\n }\n\n if (EKAM_DEBUG) {\n fprintf(stderr, \"TCMalloc workaround. Bypassing Ekam for %s\\n\", pathname);\n }\n return syscall(SYS_openat, AT_FDCWD, pathname, flags, mode);\n }\n#endif\n\n if (real_open == NULL) {\n real_open = (open_t*) dlsym(RTLD_NEXT, \"open\");\n assert(real_open != NULL);\n }\n\n remapped = remap_file(\"open\", pathname, buffer, (flags & O_ACCMODE) == O_RDONLY ? READ : WRITE);\n if (remapped == NULL) return -1;\n\n if (flags & O_CREAT) {\n mode_t mode = va_arg(args, int);\n return real_open(remapped, flags, mode);\n } else {\n return real_open(remapped, flags, 0);\n }\n}\n\nint open(const char * pathname, int flags, ...) {\n va_list args;\n va_start(args, flags);\n return intercepted_open(pathname, flags, args);\n}\n\nint _open(const char * pathname, int flags, ...) {\n va_list args;\n va_start(args, flags);\n return intercepted_open(pathname, flags, args);\n}\n\nstatic int intercepted_open64(const char * pathname, int flags, va_list args) {\n static open_t* real_open64;\n char buffer[PATH_MAX];\n const char* remapped;\n\n if (real_open64 == NULL) {\n real_open64 = (open_t*) dlsym(RTLD_NEXT, \"open64\");\n assert(real_open64 != NULL);\n }\n\n remapped = remap_file(\"open64\", pathname, buffer, (flags & O_ACCMODE) == O_RDONLY ? READ : WRITE);\n if (remapped == NULL) return -1;\n\n if(flags & O_CREAT) {\n mode_t mode = va_arg(args, int);\n return real_open64(remapped, flags, mode);\n } else {\n return real_open64(remapped, flags, 0);\n }\n}\n\nint open64(const char * pathname, int flags, ...) {\n va_list args;\n va_start(args, flags);\n return intercepted_open64(pathname, flags, args);\n}\n\nint _open64(const char * pathname, int flags, ...) {\n va_list args;\n va_start(args, flags);\n return intercepted_open64(pathname, flags, args);\n}\n\nstatic int intercepted_openat(int dirfd, const char * pathname, int flags, va_list args) {\n#if __linux__\n /* We're going to emulate openat() on top of open() by finding out the full path of the directory\n * via /proc/self/fd. */\n char procfile[128];\n char buffer[PATH_MAX];\n ssize_t n;\n\n if (pathname[0] == '/') {\n return intercepted_open(pathname, flags, args);\n }\n\n sprintf(procfile, \"/proc/self/fd/%d\", dirfd);\n n = readlink(procfile, buffer, sizeof(buffer));\n if (n < 0) {\n return n;\n }\n if (n + strlen(pathname) + 2 >= sizeof(buffer)) {\n errno = ENAMETOOLONG;\n return -1;\n }\n\n buffer[n] = '/';\n strcpy(buffer + n + 1, pathname);\n return intercepted_open(buffer, flags, args);\n#else\n fprintf(stderr, \"openat(%s) not intercepted\\n\", pathname);\n errno = ENOSYS;\n return -1;\n#endif\n}\n\nint openat(int dirfd, const char * pathname, int flags, ...) {\n va_list args;\n va_start(args, flags);\n return intercepted_openat(dirfd, pathname, flags, args);\n}\n\nint _openat(int dirfd, const char * pathname, int flags, ...) {\n va_list args;\n va_start(args, flags);\n return intercepted_openat(dirfd, pathname, flags, args);\n}\n\nstatic int intercepted_openat64(int dirfd, const char * pathname, int flags, va_list args) {\n#if __linux__\n /* We're going to emulate openat() on top of open() by finding out the full path of the directory\n * via /proc/self/fd. */\n char procfile[128];\n char buffer[PATH_MAX];\n ssize_t n;\n\n if (pathname[0] == '/') {\n return intercepted_open64(pathname, flags, args);\n }\n\n sprintf(procfile, \"/proc/self/fd/%d\", dirfd);\n n = readlink(procfile, buffer, sizeof(buffer));\n if (n < 0) {\n return n;\n }\n if (n + strlen(pathname) + 2 >= sizeof(buffer)) {\n errno = ENAMETOOLONG;\n return -1;\n }\n\n buffer[n] = '/';\n strcpy(buffer + n + 1, pathname);\n return intercepted_open64(buffer, flags, args);\n#else\n fprintf(stderr, \"openat(%s) not intercepted\\n\", pathname);\n errno = ENOSYS;\n return -1;\n#endif\n}\n\nint openat64(int dirfd, const char * pathname, int flags, ...) {\n va_list args;\n va_start(args, flags);\n return intercepted_openat64(dirfd, pathname, flags, args);\n}\n\nint _openat64(int dirfd, const char * pathname, int flags, ...) {\n va_list args;\n va_start(args, flags);\n return intercepted_openat64(dirfd, pathname, flags, args);\n}\n\n/* For rename(), we consider both locations to be outputs, since both are modified. */\ntypedef int rename_t(const char* from, const char* to);\nint rename(const char* from, const char* to) {\n static rename_t* real_rename = NULL;\n char buffer[PATH_MAX];\n char buffer2[PATH_MAX];\n\n if (real_rename == NULL) {\n real_rename = (rename_t*) dlsym(RTLD_NEXT, \"rename\");\n assert(real_rename != NULL);\n }\n\n from = remap_file(\"rename\", from, buffer, WRITE);\n if (from == NULL) return -1;\n to = remap_file(\"rename\", to, buffer2, WRITE);\n if (to == NULL) return -1;\n return real_rename (from, to);\n}\nint _rename(const char* from, const char* to) {\n return rename(from, to);\n}\n\n/* We override mkdir to just make the directory under tmp/. Ekam doesn't really care to track\n * directory creations. */\ntypedef int mkdir_t(const char* path, mode_t mode);\nint mkdir(const char* path, mode_t mode) {\n static mkdir_t* real_mkdir = NULL;\n char buffer[PATH_MAX];\n char* slash_pos;\n\n if (real_mkdir == NULL) {\n real_mkdir = (mkdir_t*) dlsym(RTLD_NEXT, \"mkdir\");\n assert(real_mkdir != NULL);\n }\n\n if (*path == '/') {\n /* Absolute path. Use the regular remap logic (which as of this writing will only allow\n * writes to /tmp). */\n path = remap_file(\"mkdir\", path, buffer, WRITE);\n if (path == NULL) return -1;\n return real_mkdir(path, mode);\n } else {\n strcpy(buffer, \"tmp/\");\n if (strlen(path) + strlen(buffer) + 1 > PATH_MAX) {\n errno = ENAMETOOLONG;\n return -1;\n }\n\n strcat(buffer, path);\n\n /* Attempt to create parent directories. */\n slash_pos = buffer;\n while ((slash_pos = strchr(slash_pos, '/')) != NULL) {\n *slash_pos = '\\0';\n /* We don't care if this fails -- we'll find out when we make the final mkdir call below. */\n real_mkdir(buffer, mode);\n *slash_pos = '/';\n ++slash_pos;\n }\n\n /* Finally create the requested directory. */\n return real_mkdir(buffer, mode);\n }\n}\nint _mkdir(const char* path, mode_t mode) {\n return mkdir(path, mode);\n}\n\n/* HACK: Ekam doesn't really track directories right now, but some tools call access() on\n * directories. Here, we check if the path exists as a directory in src or tmp and, if so,\n * go ahead and return success. Otherwise, we remap as normal.\n * TODO: Add some sort of directory handling to Ekam? */\ntypedef int access_t(const char* path, int mode);\nint access(const char* path, int mode) {\n static access_t* real_access = NULL;\n char buffer[PATH_MAX];\n struct stat stats;\n\n if (real_access == NULL) {\n real_access = (access_t*) dlsym(RTLD_NEXT, \"access\");\n assert(real_access != NULL);\n }\n\n if (*path != '/') {\n strcpy(buffer, \"src/\");\n\n if (strlen(path) + strlen(buffer) + 1 > PATH_MAX) {\n errno = ENAMETOOLONG;\n return -1;\n }\n\n strcat(buffer, path);\n if (direct_stat(buffer, &stats) == 0 && S_ISDIR(stats.st_mode)) {\n /* Directory exists in src. */\n return 0;\n }\n\n memcpy(buffer, \"tmp\", 3);\n if (direct_stat(buffer, &stats) == 0 && S_ISDIR(stats.st_mode)) {\n /* Directory exists in tmp. */\n return 0;\n }\n\n path = remap_file(\"access\", path, buffer, READ);\n if (path == NULL) return -1;\n /* TODO: If checking W_OK, we should probably only succeed if this program created the file\n * in the first place. For the moment we simply disallow writing to source files, as a\n * compromise. */\n if ((mode & W_OK) && strncmp(path, \"src/\", 4) == 0) {\n /* Cannot write to source files. */\n errno = EACCES;\n return -1;\n }\n return real_access(path, mode);\n } else {\n /* Absolute path. Don't do anything fancy. */\n path = remap_file(\"access\", path, buffer, (mode & W_OK) ? WRITE : READ);\n if (path == NULL) return -1;\n return real_access(path, mode);\n }\n}\n" }, { "path": "src/ekam/rules/intercept.ekam-rule", "content": "#! /bin/sh\n\n# Ekam Build System\n# Author: Kenton Varda (kenton@sandstorm.io)\n# Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# This is a one-off rule that builds intercept.so from intercept.c.\n\nset -eu\n\nINPUT=intercept.c\necho findProvider file:intercept.c\nread INPUT_DISK_PATH\n\nhandle_target() {\n OUTPUT=\"${INPUT%.c}$1.so\"\n echo newOutput \"$OUTPUT\"\n read OUTPUT_DISK_PATH\n\n if test `uname` = Linux; then\n ${2}gcc -g -shared -Wall -Wl,-no-undefined -fPIC -o \"$OUTPUT_DISK_PATH\" \"$INPUT_DISK_PATH\" -ldl\n else\n ${2}gcc -g -shared -Wall -fPIC -o \"$OUTPUT_DISK_PATH\" \"$INPUT_DISK_PATH\"\n fi\n\n echo provide $OUTPUT_DISK_PATH special:ekam-interceptor$3\n}\n\nhandle_target \"\" \"\" \"\"\n\nfor TARGET in ${CROSS_TARGETS:-}; do\n handle_target .${TARGET} ${TARGET}- -${TARGET}\ndone\n" }, { "path": "src/ekam/rules/proto.ekam-rule", "content": "#! /bin/sh\n\n# Ekam Build System\n# Author: Kenton Varda (kenton@sandstorm.io)\n# Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -eu\n\nif test $# = 0; then\n echo trigger filetype:.proto\n exit 0\nfi\n\nINPUT=$1\n\nPROTO_NAME=$INPUT\nPROTO_NAME=${PROTO_NAME##*/src/}\nPROTO_NAME=${PROTO_NAME#src/}\nPROTO_NAME=${PROTO_NAME##*/include/}\nPROTO_NAME=${PROTO_NAME#include/}\n\nif test \"$PROTO_NAME\" = \"google/protobuf/descriptor.proto\" -o \\\n \"$PROTO_NAME\" = \"google/protobuf/compiler/plugin.proto\"; then\n # HACK: The generated code for this proto is checked into the repository because protoc itself\n # depends on it.\n # TODO: This could be made more general by checking if the output already exists. Or maybe\n # Ekam itself should detect when the same file exists from two sources and produce an error\n # if and only if they are not identical. That would actually be ideal here because it would\n # mean an error is produced if the checked-in generated code is out-of-date.\n exit 0\nfi\n\nif test \"$PROTO_NAME\" = \"$INPUT\"; then\n SOURCE_ROOT=.\nelse\n SOURCE_ROOT=${INPUT%/$PROTO_NAME}\nfi\n\necho findProvider special:ekam-interceptor\nread INTERCEPTOR\n\nif test \"$INTERCEPTOR\" = \"\"; then\n echo \"error: couldn't find intercept.so.\" >&2\n exit 1\nfi\n\nif test \"${PROTOC:-}\" = \"\"; then\n # No PROTOC specified; try to find one built from the source tree.\n echo findProvider file:google/protobuf/compiler/main\n read PROTOC\n\n if test \"$PROTOC\" = \"\"; then\n echo \"error: couldn't find protoc.\" >&2\n exit 1\n fi\nfi\n\nLD_PRELOAD=$INTERCEPTOR DYLD_FORCE_FLAT_NAMESPACE= DYLD_INSERT_LIBRARIES=$INTERCEPTOR \\\n$PROTOC -I\"$SOURCE_ROOT\" -I/ekam-provider/protobuf --cpp_out=\"$SOURCE_ROOT\" \"$INPUT\" 3>&1 4<&0 >&2\n\necho findInput $INPUT\nread INPUT_DISK_PATH\n\necho provide \"$INPUT_DISK_PATH\" protobuf:\"$PROTO_NAME\"\n" }, { "path": "src/ekam/rules/test.ekam-rule", "content": "#! /bin/sh\n\n# Ekam Build System\n# Author: Kenton Varda (kenton@sandstorm.io)\n# Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -eu\n\nif test $# = 0; then\n echo trigger test:executable\n exit 0\nfi\n\necho findInput \"$1\"\nread TEST_PROG\n\nINTERCEPTOR_TAG=special:ekam-interceptor\nINTERPRETER=\n\nfor TARGET in ${CROSS_TARGETS:-}; do\n if test \"${1%.$TARGET}\" != \"$1\"; then\n INTERCEPTOR_TAG=\"special:ekam-interceptor-$TARGET\"\n export QEMU_LD_PREFIX=/usr/$TARGET\n INTERPRETER=\"qemu-${TARGET%-linux-gnu}\"\n fi\ndone\n\nif grep -q EKAM_TEST_DISABLE_INTERCEPTOR \"$TEST_PROG\"; then\n # Test requested no interceptor.\n INTERCEPTOR=\nelse\n echo findProvider \"$INTERCEPTOR_TAG\"\n read INTERCEPTOR\n\n if test \"$INTERCEPTOR\" = \"\"; then\n echo \"error: couldn't find intercept.so.\" >&2\n exit 1\n fi\nfi\n\n# TODO: Run in sandbox.\n\necho newOutput \"${1}.log\"\nread TEST_LOG\n\nif ${TEST_WRAPPER:-} env LD_PRELOAD=$INTERCEPTOR DYLD_FORCE_FLAT_NAMESPACE= DYLD_INSERT_LIBRARIES=$INTERCEPTOR \\\n $INTERPRETER $TEST_PROG 3>&1 4<&0 2>\"$TEST_LOG\" >&2; then\n echo passed\n exit 0\nelse\n echo \"full log: $TEST_LOG\" >&2\n egrep 'FAIL|ERROR|FATAL|ekam-provider' \"$TEST_LOG\" >&2\n exit 1\nfi\n" }, { "path": "src/os/ByteStream.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"ByteStream.h\"\n\n#include \n#include \n#include \n#include \n#include \n#include \n\n#include \"base/Debug.h\"\n\nnamespace ekam {\n\nByteStream::ByteStream(const std::string& path, int flags)\n : handle(path, WRAP_SYSCALL(open, path.c_str(), flags, 0666)) {}\n\nByteStream::ByteStream(const std::string& path, int flags, int mode)\n : handle(path, WRAP_SYSCALL(open, path.c_str(), flags, mode)) {}\n\nByteStream::ByteStream(int fd, const std::string& name)\n : handle(name, fd) {}\n\nByteStream::~ByteStream() noexcept(false) {}\n\nsize_t ByteStream::read(void* buffer, size_t size) {\n return WRAP_SYSCALL(read, handle, buffer, size);\n}\n\nPromise ByteStream::readAsync(EventManager* eventManager, void* buffer, size_t size) {\n if (watcher == nullptr) {\n watcher = eventManager->watchFd(handle.get());\n }\n return eventManager->when(watcher->onReadable())(\n [this, buffer, size](Void) -> size_t {\n return read(buffer, size);\n });\n}\n\nsize_t ByteStream::write(const void* buffer, size_t size) {\n return WRAP_SYSCALL(write, handle, buffer, size);\n}\n\nvoid ByteStream::writeAll(const void* buffer, size_t size) {\n const char* cbuffer = reinterpret_cast(buffer);\n\n while (size > 0) {\n ssize_t n = write(cbuffer, size);\n cbuffer += n;\n size -= n;\n }\n}\n\nvoid ByteStream::stat(struct stat* stats) {\n WRAP_SYSCALL(fstat, handle, stats);\n}\n\n// =======================================================================================\n\nPipe::Pipe() {\n if (pipe(fds) != 0) {\n throw OsError(\"\", \"pipe\", errno);\n }\n\n fcntl(fds[0], F_SETFD, FD_CLOEXEC);\n fcntl(fds[1], F_SETFD, FD_CLOEXEC);\n}\n\nPipe::~Pipe() {\n closeReadEnd();\n closeWriteEnd();\n}\n\nOwnedPtr Pipe::releaseReadEnd() {\n auto result = newOwned(fds[0], \"pipe.readEnd\");\n fds[0] = -1;\n return result.release();\n}\n\nOwnedPtr Pipe::releaseWriteEnd() {\n auto result = newOwned(fds[1], \"pipe.writeEnd\");\n fds[1] = -1;\n return result.release();\n}\n\nvoid Pipe::attachReadEndForExec(int target) {\n dup2(fds[0], target);\n closeReadEnd();\n closeWriteEnd();\n}\n\nvoid Pipe::attachWriteEndForExec(int target) {\n dup2(fds[1], target);\n closeReadEnd();\n closeWriteEnd();\n}\n\nvoid Pipe::closeReadEnd() {\n if (fds[0] != -1) {\n if (close(fds[0]) != 0) {\n DEBUG_ERROR << \"close(pipe): \" << strerror(errno);\n }\n fds[0] = -1;\n }\n}\n\nvoid Pipe::closeWriteEnd() {\n if (fds[1] != -1) {\n if (close(fds[1]) != 0) {\n DEBUG_ERROR << \"close(pipe): \" << strerror(errno);\n }\n fds[1] = -1;\n }\n}\n\n} // namespace ekam\n" }, { "path": "src/os/ByteStream.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_OS_BYTESTREAM_H_\n#define KENTONSCODE_OS_BYTESTREAM_H_\n\n#include \n#include \n#include \n\n#include \"base/OwnedPtr.h\"\n#include \"OsHandle.h\"\n#include \"EventManager.h\"\n\nnamespace ekam {\n\nclass EventManager;\n\nclass ByteStream {\npublic:\n // TODO(kenton): Make class abstract.\n ByteStream(const std::string& path, int flags);\n ByteStream(const std::string& path, int flags, int mode);\n ByteStream(int fd, const std::string& name);\n ~ByteStream() noexcept(false);\n\n inline OsHandle* getHandle() { return &handle; }\n\n size_t read(void* buffer, size_t size);\n Promise readAsync(EventManager* eventManager, void* buffer, size_t size);\n size_t write(const void* buffer, size_t size);\n void writeAll(const void* buffer, size_t size);\n void stat(struct stat* stats);\n\nprivate:\n class ReadEventCallback;\n\n OsHandle handle;\n OwnedPtr watcher;\n};\n\nclass Pipe {\npublic:\n Pipe();\n ~Pipe();\n\n OwnedPtr releaseReadEnd();\n OwnedPtr releaseWriteEnd();\n void attachReadEndForExec(int target);\n void attachWriteEndForExec(int target);\n\nprivate:\n int fds[2];\n\n void closeReadEnd();\n void closeWriteEnd();\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_OS_BYTESTREAM_H_\n" }, { "path": "src/os/DiskFile.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"DiskFile.h\"\n\n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n\n#include \"base/Debug.h\"\n#include \"os/OsHandle.h\"\n#include \"os/ByteStream.h\"\n#include \"base/Hash.h\"\n\nnamespace ekam {\n\n// TODO: Use new openat() and friends? Very capability-like!\n\nnamespace {\n\nclass DirectoryReader {\npublic:\n DirectoryReader(const std::string& path)\n : path(path),\n dir(opendir(path.c_str())) {\n if (dir == NULL) {\n // TODO: Better exception type.\n throw OsError(path, \"opendir\", ENOTDIR);\n }\n }\n\n ~DirectoryReader() {\n if (closedir(dir) < 0) {\n DEBUG_ERROR << path << \": closedir: \" << strerror(errno);\n }\n }\n\n bool next(std::string* output) {\n errno = 0;\n struct dirent *entryPointer = readdir(dir);\n if (errno != 0) {\n throw OsError(path, \"readdir\", errno);\n }\n\n if (entryPointer == NULL) {\n return false;\n } else {\n *output = entryPointer->d_name;\n return true;\n }\n }\n\nprivate:\n const std::string& path;\n DIR* dir;\n};\n\nbool statIfExists(const std::string& path, struct stat* output) {\n int result;\n do {\n result = stat(path.c_str(), output);\n } while (result < 0 && errno == EINTR);\n\n if (result == 0) {\n return true;\n } else if (errno == ENOENT) {\n return false;\n } else {\n throw OsError(path, \"stat\", errno);\n }\n}\n\n} // anonymous namespace\n\nDiskFile::DiskFile(const std::string& path, File* parent) : path(path) {\n if (parent != NULL) {\n this->parentRef = parent->clone();\n }\n}\nDiskFile::~DiskFile() {}\n\nstd::string DiskFile::basename() {\n if (path.empty()) {\n return \".\";\n }\n\n std::string::size_type slashPos = path.find_last_of('/');\n if (slashPos == std::string::npos) {\n return path;\n } else {\n return path.substr(slashPos + 1);\n }\n}\n\nstd::string DiskFile::canonicalName() {\n if (parentRef == NULL) {\n return \".\";\n }\n\n std::string result = parentRef->canonicalName();\n if (result == \".\") {\n result.clear();\n } else {\n result.push_back('/');\n }\n result.append(basename());\n\n return result;\n}\n\nOwnedPtr DiskFile::clone() {\n return newOwned(path, parentRef.get());\n}\n\nbool DiskFile::hasParent() {\n return parentRef != NULL;\n}\n\nOwnedPtr DiskFile::parent() {\n if (parentRef == NULL) {\n throw std::runtime_error(\"Tried to get parent of top-level directory: \" + canonicalName());\n }\n return parentRef->clone();\n}\n\nbool DiskFile::equals(File* other) {\n DiskFile* otherDiskFile = dynamic_cast(other);\n return otherDiskFile != NULL && otherDiskFile->path == path;\n}\n\nsize_t DiskFile::identityHash() {\n return std::hash()(path);\n}\n\nclass DiskFile::DiskRefImpl : public File::DiskRef {\npublic:\n DiskRefImpl(const std::string& path) : pathName(path) {}\n ~DiskRefImpl() {}\n\n // implements DiskRef ------------------------------------------------------------------\n virtual const std::string& path() { return pathName; }\n\nprivate:\n std::string pathName;\n};\n\nOwnedPtr DiskFile::getOnDisk(Usage usage) {\n return newOwned(path);\n}\n\nbool DiskFile::exists() {\n struct stat stats;\n return statIfExists(path.c_str(), &stats) && (S_ISREG(stats.st_mode) || S_ISDIR(stats.st_mode));\n}\n\nbool DiskFile::isFile() {\n struct stat stats;\n return statIfExists(path.c_str(), &stats) && S_ISREG(stats.st_mode);\n}\n\nbool DiskFile::isDirectory() {\n struct stat stats;\n return statIfExists(path.c_str(), &stats) && S_ISDIR(stats.st_mode);\n}\n\n// File only.\nHash DiskFile::contentHash() {\n try {\n Hash::Builder hasher;\n ByteStream fd(path, O_RDONLY);\n\n char buffer[8192];\n\n while (true) {\n size_t n = fd.read(buffer, sizeof(buffer));\n if (n == 0) {\n return hasher.build();\n }\n\n hasher.add(buffer, n);\n }\n } catch (const OsError& e) {\n if (e.getErrorNumber() == ENOENT || e.getErrorNumber() == EACCES ||\n e.getErrorNumber() == EISDIR) {\n return Hash::NULL_HASH;\n }\n throw;\n }\n}\n\nstd::string DiskFile::readAll() {\n ByteStream fd(path, O_RDONLY);\n\n struct stat stats;\n fd.stat(&stats);\n size_t size = stats.st_size;\n\n std::string result;\n if (size > 0) {\n result.resize(size);\n char* buffer = &*result.begin();\n size_t bytesRead = 0;\n while (size > bytesRead) {\n ssize_t n = fd.read(buffer + bytesRead, size - bytesRead);\n\n if (n == 0) {\n result.resize(bytesRead);\n size = bytesRead;\n } else {\n bytesRead += n;\n }\n }\n }\n\n return result;\n}\n\nvoid DiskFile::writeAll(const std::string& content) {\n ByteStream fd(path, O_WRONLY | O_TRUNC | O_CREAT);\n\n std::string::size_type pos = 0;\n while (pos < content.size()) {\n pos += fd.write(content.data() + pos, content.size() - pos);\n }\n}\n\nvoid DiskFile::writeAll(const void* data, int size) {\n ByteStream fd(path, O_WRONLY | O_TRUNC | O_CREAT);\n\n const char* pos = reinterpret_cast(data);\n while (size > 0) {\n int n = fd.write(pos, size);\n pos += n;\n size -= n;\n }\n}\n\n// Directory only.\nvoid DiskFile::list(OwnedPtrVector::Appender output) {\n std::string prefix;\n if (!path.empty()) {\n prefix = path + \"/\";\n }\n\n DirectoryReader reader(path);\n std::string filename;\n while (reader.next(&filename)) {\n if (filename.empty()) {\n DEBUG_ERROR << \"DirectoryReader returned empty file name.\";\n } else if (filename[0] != '.') { // skip hidden files\n output.add(newOwned(prefix + filename, this));\n }\n }\n}\n\nOwnedPtr DiskFile::relative(const std::string& path) {\n if (path.empty()) {\n throw std::invalid_argument(\"File::relative(): path cannot be empty.\");\n } else if (path[0] == '/') {\n throw std::invalid_argument(\"File::relative(): path cannot start with a slash.\");\n }\n\n std::string::size_type slash_pos = path.find_first_of('/');\n std::string first_part;\n std::string rest;\n if (slash_pos == std::string::npos) {\n if (path == \".\") {\n return clone();\n } else if (path == \"..\") {\n return parent();\n } else if (this->path.empty()) {\n return newOwned(path, this);\n } else {\n return newOwned(this->path + \"/\" + path, this);\n }\n\n } else {\n first_part.assign(path, 0, slash_pos);\n\n std::string::size_type after_slash_pos = path.find_first_not_of('/', slash_pos);\n\n if (after_slash_pos == std::string::npos) {\n // Trailing slash. Bah.\n return relative(first_part);\n } else {\n rest.assign(path, after_slash_pos, std::string::npos);\n\n if (first_part == \".\") {\n return relative(rest);\n } else if (first_part == \"..\") {\n if (parentRef == NULL) {\n throw std::runtime_error(\n \"Tried to get parent of top-level directory: \" + canonicalName());\n }\n return parentRef->relative(rest);\n } else {\n OwnedPtr temp;\n if (this->path.empty()) {\n temp = newOwned(first_part, this);\n } else {\n temp = newOwned(this->path + \"/\" + first_part, this);\n }\n return temp->relative(rest);\n }\n }\n }\n}\n\nvoid DiskFile::createDirectory() {\n while (true) {\n if (mkdir(path.c_str(), 0777) == 0) {\n return;\n } else if (errno != EINTR) {\n throw OsError(path, \"mkdir\", errno);\n }\n }\n}\n\nvoid DiskFile::link(File* target) {\n DiskFile* diskTarget = dynamic_cast(target);\n if (diskTarget == NULL) {\n throw new std::invalid_argument(\"Cannot link disk file to non-disk file: \" + path);\n }\n\n WRAP_SYSCALL(link, diskTarget->path.c_str(), path.c_str());\n}\n\nvoid DiskFile::unlink() {\n WRAP_SYSCALL(unlink, path.c_str());\n}\n\n} // namespace ekam\n" }, { "path": "src/os/DiskFile.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_OS_DISKFILE_H_\n#define KENTONSCODE_OS_DISKFILE_H_\n\n#include \"File.h\"\n#include \n\nnamespace ekam {\n\nclass DiskFile: public File {\npublic:\n DiskFile(const std::string& path, File* parent);\n ~DiskFile();\n\n // implements File ---------------------------------------------------------------------\n std::string basename();\n std::string canonicalName();\n OwnedPtr clone();\n bool hasParent();\n OwnedPtr parent();\n\n bool equals(File* other);\n size_t identityHash();\n\n OwnedPtr getOnDisk(Usage usage);\n\n bool exists();\n bool isFile();\n bool isDirectory();\n\n // File only.\n Hash contentHash();\n std::string readAll();\n void writeAll(const std::string& content);\n void writeAll(const void* data, int size);\n\n // Directory only.\n void list(OwnedPtrVector::Appender output);\n OwnedPtr relative(const std::string& path);\n\n // Methods that create or delete objects.\n void createDirectory();\n void link(File* target);\n void unlink();\n\nprivate:\n class DiskRefImpl;\n\n std::string path;\n OwnedPtr parentRef;\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_OS_DISKFILE_H_\n" }, { "path": "src/os/EpollEventManager.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"EpollEventManager.h\"\n\n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n\n#include \"base/Debug.h\"\n#include \"base/Table.h\"\n\nnamespace ekam {\n\nnamespace {\n\nstd::string epollEventsToString(uint32_t events) {\n std::string result;\n if (events == 0) {\n result.append(\" (none)\");\n }\n if (events & EPOLLIN) {\n result.append(\" EPOLLIN\");\n }\n if (events & EPOLLOUT) {\n result.append(\" EPOLLOUT\");\n }\n if (events & EPOLLRDHUP) {\n result.append(\" EPOLLRDHUP\");\n }\n if (events & EPOLLPRI) {\n result.append(\" EPOLLPRI\");\n }\n if (events & EPOLLERR) {\n result.append(\" EPOLLERR\");\n }\n if (events & EPOLLHUP) {\n result.append(\" EPOLLHUP\");\n }\n if (events & EPOLLET) {\n result.append(\" EPOLLET\");\n }\n if (events & EPOLLONESHOT) {\n result.append(\" EPOLLONESHOT\");\n }\n\n if (events & ~(EPOLLIN | EPOLLOUT | EPOLLRDHUP | EPOLLPRI | EPOLLERR | EPOLLHUP |\n EPOLLET | EPOLLONESHOT)) {\n result.append(\" (others)\");\n }\n\n return result;\n}\n\n// TODO: Copied from DiskFile.cpp. Share code somehow?\nbool statIfExists(const std::string& path, struct stat* output) {\n int result;\n do {\n result = stat(path.c_str(), output);\n } while (result < 0 && errno == EINTR);\n\n if (result == 0) {\n return true;\n } else if (errno == ENOENT) {\n return false;\n } else {\n throw OsError(path, \"stat\", errno);\n }\n}\n\nbool isDirectory(const std::string& path) {\n struct stat stats;\n return statIfExists(path.c_str(), &stats) && S_ISDIR(stats.st_mode);\n}\n\n} // namespace\n\nEpollEventManager::Epoller::Epoller()\n : epollHandle(\"epoll\", WRAP_SYSCALL(epoll_create1, (int)EPOLL_CLOEXEC)),\n watchCount(0) {}\n\nEpollEventManager::Epoller::~Epoller() {\n if (watchCount > 0) {\n DEBUG_ERROR << \"Epoller destroyed before all Watches destroyed.\";\n }\n}\n\nEpollEventManager::Epoller::Watch::Watch(Epoller* epoller, OsHandle* handle,\n uint32_t events, IoHandler* handler)\n : epoller(epoller), events(0), registeredEvents(0), fd(handle->get()), name(handle->getName()),\n handler(handler) {\n addEvents(events);\n}\n\nEpollEventManager::Epoller::Watch::Watch(Epoller* epoller, int fd,\n uint32_t events, IoHandler* handler)\n : epoller(epoller), events(0), registeredEvents(0), fd(fd), name(toString(fd)),\n handler(handler) {\n addEvents(events);\n}\n\nEpollEventManager::Epoller::Watch::~Watch() {\n removeEvents(events);\n\n if (epoller->watchesNeedingUpdate.erase(this) > 0) {\n updateRegistration();\n }\n}\n\nvoid EpollEventManager::Epoller::Watch::addEvents(uint32_t eventsToAdd) {\n DEBUG_INFO << \"Adding events for \" << fd << \":\" << epollEventsToString(eventsToAdd);\n uint32_t newEvents = events | eventsToAdd;\n if (newEvents == events) {\n return;\n }\n\n events = newEvents;\n if (events == registeredEvents) {\n epoller->watchesNeedingUpdate.erase(this);\n } else {\n epoller->watchesNeedingUpdate.insert(this);\n }\n}\n\nvoid EpollEventManager::Epoller::Watch::removeEvents(uint32_t eventsToRemove) {\n DEBUG_INFO << \"Removing events for \" << fd << \":\" << epollEventsToString(eventsToRemove);\n uint32_t newEvents = events & ~eventsToRemove;\n if (newEvents == events) {\n return;\n }\n\n events = newEvents;\n if (events == registeredEvents) {\n epoller->watchesNeedingUpdate.erase(this);\n } else {\n epoller->watchesNeedingUpdate.insert(this);\n }\n}\n\nvoid EpollEventManager::Epoller::Watch::updateRegistration() {\n if (registeredEvents == events) {\n DEBUG_ERROR << \"Watch does not need updating.\";\n return;\n }\n DEBUG_INFO << \"Updating event registrations for \" << fd << \":\" << epollEventsToString(events);\n\n int op = EPOLL_CTL_MOD;\n if (registeredEvents == 0) {\n ++epoller->watchCount;\n op = EPOLL_CTL_ADD;\n } else if (events == 0) {\n --epoller->watchCount;\n op = EPOLL_CTL_DEL;\n }\n registeredEvents = events;\n\n struct epoll_event event;\n event.events = registeredEvents;\n event.data.ptr = this;\n WRAP_SYSCALL(epoll_ctl, epoller->epollHandle, op, fd, &event);\n}\n\nbool EpollEventManager::Epoller::handleEvent() {\n // Run pending updates.\n for (Watch* watch : watchesNeedingUpdate) {\n watch->updateRegistration();\n }\n watchesNeedingUpdate.clear();\n\n if (watchCount == 0) {\n DEBUG_INFO << \"No more events.\";\n return false;\n }\n\n DEBUG_INFO << \"Waiting for \" << watchCount << \" events...\";\n struct epoll_event event;\n int result = WRAP_SYSCALL(epoll_wait, epollHandle, &event, 1, -1);\n if (result == 0) {\n throw std::logic_error(\"epoll_wait() returned zero despite infinite timeout.\");\n } else if (result > 1) {\n throw std::logic_error(\"epoll_wait() returned more than one event when only one requested.\");\n }\n\n Watch* watch = reinterpret_cast(event.data.ptr);\n DEBUG_INFO << \"epoll event: \" << watch->name << \":\" << epollEventsToString(event.events);\n\n watch->handler->handle(event.events);\n\n return true;\n}\n\n// =============================================================================\n\nnamespace {\n\nsigset_t getHandledSignals() {\n sigset_t result;\n sigemptyset(&result);\n sigaddset(&result, SIGCHLD);\n return result;\n}\n\nconst sigset_t HANDLED_SIGNALS = getHandledSignals();\n\n} // namespace\n\nEpollEventManager::SignalHandler::SignalHandler(Epoller* epoller)\n : signalStream(WRAP_SYSCALL(signalfd, -1, &HANDLED_SIGNALS, SFD_NONBLOCK | SFD_CLOEXEC),\n \"signalfd\"),\n watch(epoller, signalStream.getHandle(), 0, this) {\n sigprocmask(SIG_BLOCK, &HANDLED_SIGNALS, NULL);\n}\n\nEpollEventManager::SignalHandler::~SignalHandler() {\n sigprocmask(SIG_UNBLOCK, &HANDLED_SIGNALS, NULL);\n}\n\nvoid EpollEventManager::SignalHandler::handle(uint32_t events) {\n DEBUG_INFO << \"Received signal on signalfd.\";\n\n struct signalfd_siginfo signalEvent;\n if (signalStream.read(&signalEvent, sizeof(signalEvent)) != sizeof(signalEvent)) {\n DEBUG_ERROR << \"read(signalfd) returned wrong size.\";\n return;\n }\n\n if (signalEvent.ssi_signo == SIGCHLD) {\n // Alas, the contents of signalEvent are useless, as the signal queue only holds one signal\n // per signal number. Therefore, if two SIGCHLDs are received before we have a chance to\n // handle the first one, one of the two is discarded. But we have to wait() to avoid zombies\n // anyway so I guess it's no big deal.\n handleProcessExit();\n } else {\n DEBUG_ERROR << \"Unexpected signal number: \" << signalEvent.ssi_signo;\n }\n}\n\nvoid EpollEventManager::SignalHandler::maybeStopExpecting() {\n if (processExitHandlerMap.empty()) {\n watch.removeEvents(EPOLLIN);\n }\n}\n\n// =======================================================================================\n\nclass EpollEventManager::AsyncCallbackHandler : public PendingRunnable {\npublic:\n AsyncCallbackHandler(EpollEventManager* eventManager, OwnedPtr runnable)\n : eventManager(eventManager), called(false), runnable(runnable.release()) {\n eventManager->asyncCallbacks.push_back(this);\n }\n ~AsyncCallbackHandler() {\n if (!called) {\n for (std::deque::iterator iter = eventManager->asyncCallbacks.begin();\n iter != eventManager->asyncCallbacks.end(); ++iter) {\n if (*iter == this) {\n eventManager->asyncCallbacks.erase(iter);\n return;\n }\n }\n DEBUG_ERROR << \"AsyncCallbackHandler not called but not in asyncCallbacks.\";\n }\n }\n\n void run() {\n called = true;\n runnable->run();\n }\n\nprivate:\n EpollEventManager* eventManager;\n bool called;\n OwnedPtr runnable;\n};\n\nOwnedPtr EpollEventManager::runLater(OwnedPtr runnable) {\n return newOwned(this, runnable.release());\n}\n\n// =======================================================================================\n\nclass EpollEventManager::SignalHandler::ProcessExitHandler\n : public PromiseFulfiller {\npublic:\n ProcessExitHandler(Callback* callback, SignalHandler* signalHandler, pid_t pid)\n : callback(callback), signalHandler(signalHandler), pid(pid) {\n if (!signalHandler->processExitHandlerMap.insert(\n std::make_pair(pid, this)).second) {\n throw std::runtime_error(\"Already waiting on this process.\");\n }\n signalHandler->watch.addEvents(EPOLLIN);\n }\n ~ProcessExitHandler() {\n if (pid != -1) {\n signalHandler->processExitHandlerMap.erase(pid);\n signalHandler->maybeStopExpecting();\n }\n }\n\n void handle(int waitStatus) {\n DEBUG_INFO << \"Process \" << pid << \" exited with status: \" << waitStatus;\n\n signalHandler->processExitHandlerMap.erase(pid);\n signalHandler->maybeStopExpecting();\n pid = -1;\n\n if (WIFEXITED(waitStatus)) {\n callback->fulfill(ProcessExitCode(WEXITSTATUS(waitStatus)));\n } else if (WIFSIGNALED(waitStatus)) {\n callback->fulfill(ProcessExitCode(ProcessExitCode::SIGNALED, WTERMSIG(waitStatus)));\n } else {\n DEBUG_ERROR << \"Didn't understand process exit status.\";\n callback->fulfill(ProcessExitCode(-1));\n }\n }\n\nprivate:\n Callback* callback;\n SignalHandler* signalHandler;\n pid_t pid;\n};\n\nvoid EpollEventManager::SignalHandler::handleProcessExit() {\n // If multiple signals with the same signal number are delivered while signals are blocked,\n // only one of them is actually delivered once un-blocked. The others are cast into the\n // void. Therefore, the contents of the siginfo structure are effectively useless for\n // SIGCHLD. We must instead call waitpid() repeatedly until there are no more completed\n // children. Signals suck so much.\n while (true) {\n int waitStatus;\n pid_t pid = waitpid(-1, &waitStatus, WNOHANG);\n if (pid < 0) {\n // ECHILD indicates there are no child processes. Anything else is a real error.\n if (errno != ECHILD) {\n DEBUG_ERROR << \"waitpid: \" << strerror(errno);\n }\n break;\n } else if (pid == 0) {\n // There are child processes, but they are still running.\n break;\n }\n\n // Get the handler associated with this PID.\n std::unordered_map::iterator iter =\n processExitHandlerMap.find(pid);\n if (iter == processExitHandlerMap.end()) {\n // It is actually important that any code creating a subprocess call onProcessExit()\n // to receive notification of completion even if it doesn't care about completion,\n // because otherwise the sub-process may be stuck as a zombie. This is actually\n // NOT the case with EpollEventManager because it waits on all sub-processes whether\n // onProcessExit() was called or not, but we should warn if we encounter a process for\n // which onProcessExit() was never called so that the code can be fixed.\n DEBUG_ERROR << \"Got SIGCHLD for PID we weren't waiting for: \" << pid;\n return;\n }\n\n iter->second->handle(waitStatus);\n }\n}\n\nPromise EpollEventManager::SignalHandler::onProcessExit(pid_t pid) {\n return newPromise(this, pid);\n}\n\nPromise EpollEventManager::onProcessExit(pid_t pid) {\n return signalHandler.onProcessExit(pid);\n}\n\n// =======================================================================================\n\nclass EpollEventManager::IoWatcherImpl: public IoWatcher, public IoHandler {\npublic:\n IoWatcherImpl(Epoller* epoller, int fd)\n : watch(epoller, fd, 0, this),\n readFulfiller(nullptr), writeFulfiller(nullptr) {}\n\n ~IoWatcherImpl() {\n if (readFulfiller != nullptr) {\n readFulfiller->abandon();\n }\n if (writeFulfiller != nullptr) {\n writeFulfiller->abandon();\n }\n }\n\n // implements IoWatcher --------------------------------------------------------------\n\n Promise onReadable() {\n if (readFulfiller != nullptr) {\n throw std::logic_error(\"Already waiting for readability on this fd.\");\n }\n return newPromise(&watch, EPOLLIN, &readFulfiller);\n }\n\n Promise onWritable() {\n if (readFulfiller != nullptr) {\n throw std::logic_error(\"Already waiting for writability on this fd.\");\n }\n return newPromise(&watch, EPOLLOUT, &writeFulfiller);\n }\n\n // implements IoHandler --------------------------------------------------------------\n void handle(uint32_t events) {\n if (events & (EPOLLIN | EPOLLERR | EPOLLHUP)) {\n if (readFulfiller != nullptr) {\n readFulfiller->ready();\n }\n }\n if (events & (EPOLLOUT | EPOLLERR | EPOLLHUP)) {\n if (writeFulfiller != nullptr) {\n writeFulfiller->ready();\n }\n }\n }\n\nprivate:\n class Fulfiller: public PromiseFulfiller {\n public:\n Fulfiller(Callback* callback, Epoller::Watch* watch, uint32_t events, Fulfiller** ptr)\n : callback(callback), watch(watch), events(events), ptr(ptr) {\n *ptr = this;\n watch->addEvents(events);\n }\n ~Fulfiller() {\n if (ptr != nullptr) {\n *ptr = nullptr;\n watch->removeEvents(events);\n }\n }\n\n void ready() {\n *ptr = nullptr;\n ptr = nullptr;\n watch->removeEvents(events);\n callback->fulfill();\n }\n\n void abandon() {\n *ptr = nullptr;\n ptr = nullptr;\n watch->removeEvents(events);\n try {\n throw std::logic_error(\"IoWatcher deleted while waiting for I/O.\");\n } catch (...) {\n callback->propagateCurrentException();\n }\n }\n\n private:\n Callback* callback;\n Epoller::Watch* watch;\n uint32_t events;\n Fulfiller** ptr;\n };\n\n Epoller::Watch watch;\n Fulfiller* readFulfiller;\n Fulfiller* writeFulfiller;\n};\n\nOwnedPtr EpollEventManager::watchFd(int fd) {\n return newOwned(&epoller, fd);\n}\n\n// =======================================================================================\n\nclass EpollEventManager::InotifyHandler::WatchedDirectory {\n class CallbackTable : public Table,\n UniqueColumn > {\n public:\n static const int BASENAME = 0;\n static const int WATCH_OP = 1;\n };\n\npublic:\n WatchedDirectory(InotifyHandler* inotifyHandler, const std::string& path)\n : inotifyHandler(inotifyHandler), path(path) {\n wd = WRAP_SYSCALL(inotify_add_watch, *inotifyHandler->inotifyStream.getHandle(), path.c_str(),\n IN_ATTRIB | IN_CLOSE_WRITE | IN_CREATE | IN_DELETE | IN_DELETE_SELF |\n IN_MODIFY | IN_MOVE_SELF | IN_MOVED_FROM | IN_MOVED_TO);\n DEBUG_INFO << \"inotify_add_watch(\" << path << \") [\" << wd << \"]\";\n inotifyHandler->watchMap[wd] = this;\n inotifyHandler->watchByNameMap[path] = this;\n inotifyHandler->watch.addEvents(EPOLLIN);\n }\n\n ~WatchedDirectory() {\n DEBUG_INFO << \"~WatchedDirectory(): \" << path;\n\n if (callbackTable.size() > 0) {\n DEBUG_ERROR << \"Deleting WatchedDirectory before all FileWatcherImpls were removed.\";\n }\n\n if (wd >= 0) {\n DEBUG_INFO << \"inotify_rm_watch(\" << path << \") [\" << wd << \"]\";\n\n if (WRAP_SYSCALL(inotify_rm_watch, *inotifyHandler->inotifyStream.getHandle(), wd) < 0) {\n DEBUG_ERROR << \"inotify_rm_watch(\" << path << \"): \" << strerror(errno);\n }\n }\n\n invalidate();\n }\n\n void addWatch(const std::string& basename, FileWatcherImpl* op) {\n DEBUG_INFO << \"Watch directory \" << path << \" now covering: \" << basename;\n callbackTable.add(basename, op);\n }\n\n void removeWatch(FileWatcherImpl* op) {\n DEBUG_INFO << \"Watch directory \" << path << \" no longer covering: \" << basenameForOp(op);\n if (callbackTable.erase(op) == 0) {\n DEBUG_ERROR << \"Trying to remove watch that was never added.\";\n }\n deleteSelfIfEmpty();\n }\n\n void invalidate() {\n if (wd >= 0) {\n inotifyHandler->watchMap.erase(wd);\n inotifyHandler->watchByNameMap.erase(path);\n wd = -1;\n }\n }\n\n void handle(struct inotify_event* event);\n\nprivate:\n InotifyHandler* inotifyHandler;\n int wd;\n std::string path;\n\n CallbackTable callbackTable;\n\n void deleteSelfIfEmpty() {\n if (callbackTable.size() == 0) {\n auto inotifyHandler = this->inotifyHandler;\n inotifyHandler->ownedWatchDirectories.erase(this);\n if (inotifyHandler->ownedWatchDirectories.empty()) {\n inotifyHandler->watch.removeEvents(EPOLLIN);\n }\n }\n }\n\n std::string basenameForOp(FileWatcherImpl* op) {\n const CallbackTable::Row* row = callbackTable.find(op);\n if (row == NULL) {\n return \"(invalid)\";\n } else {\n return row->cell();\n }\n }\n};\n\nclass EpollEventManager::InotifyHandler::FileWatcherImpl: public FileWatcher {\npublic:\n FileWatcherImpl(InotifyHandler* inotifyHandler, const std::string& filename)\n : watchedDirectory(nullptr), modified(false), deleted(false), fulfiller(nullptr) {\n // Split directory and basename.\n std::string directory;\n std::string basename;\n if (isDirectory(filename)) {\n directory = filename;\n } else {\n std::string::size_type slashPos = filename.find_last_of('/');\n if (slashPos == std::string::npos) {\n directory.assign(\".\");\n basename.assign(filename);\n } else {\n directory.assign(filename, 0, slashPos);\n basename.assign(filename, slashPos + 1, std::string::npos);\n }\n }\n\n // Find or create WatchedDirectory object.\n WatchByNameMap::iterator iter = inotifyHandler->watchByNameMap.find(directory);\n if (iter == inotifyHandler->watchByNameMap.end()) {\n auto newWatchedDirectory = newOwned(inotifyHandler, directory);\n watchedDirectory = newWatchedDirectory.get();\n inotifyHandler->ownedWatchDirectories.add(watchedDirectory, newWatchedDirectory.release());\n } else {\n watchedDirectory = iter->second;\n }\n\n // Add to WatchedDirectory.\n watchedDirectory->addWatch(basename, this);\n }\n\n ~FileWatcherImpl() {\n if (watchedDirectory != NULL) {\n watchedDirectory->removeWatch(this);\n }\n if (fulfiller != nullptr) {\n fulfiller->abandon();\n }\n }\n\n void flagAsModified() {\n modified = true;\n maybeFulfill();\n }\n\n void flagAsDeleted() {\n deleted = true;\n maybeFulfill();\n }\n\n // implements FileWatcher --------------------------------------------------------------\n Promise onChange() {\n if (fulfiller != nullptr) {\n fulfiller->abandon();\n }\n auto result = newPromise(&fulfiller);\n maybeFulfill();\n return result;\n }\n\nprivate:\n class Fulfiller: public PromiseFulfiller {\n public:\n Fulfiller(Callback* callback, Fulfiller** ptr)\n : callback(callback), ptr(ptr) {\n *ptr = this;\n }\n ~Fulfiller() {\n if (ptr != nullptr) {\n *ptr = nullptr;\n }\n }\n\n void fulfill(FileChangeType type) {\n *ptr = nullptr;\n ptr = nullptr;\n callback->fulfill(type);\n }\n\n void abandon() {\n *ptr = nullptr;\n ptr = nullptr;\n try {\n throw std::logic_error(\"FileWatcher deleted while waiting for changes.\");\n } catch (...) {\n callback->propagateCurrentException();\n }\n }\n\n private:\n Callback* callback;\n Fulfiller** ptr;\n };\n\n WatchedDirectory* watchedDirectory;\n bool modified;\n bool deleted;\n Fulfiller* fulfiller;\n\n void maybeFulfill() {\n if (fulfiller != nullptr) {\n if (deleted) {\n fulfiller->fulfill(FileChangeType::DELETED);\n } else if (modified) {\n fulfiller->fulfill(FileChangeType::MODIFIED);\n }\n deleted = false;\n modified = false;\n }\n }\n};\n\nEpollEventManager::InotifyHandler::InotifyHandler(Epoller* epoller)\n : inotifyStream(WRAP_SYSCALL(inotify_init1, IN_NONBLOCK | IN_CLOEXEC), \"inotify\"),\n watch(epoller, inotifyStream.getHandle(), 0, this) {}\n\nEpollEventManager::InotifyHandler::~InotifyHandler() {}\n\nvoid EpollEventManager::InotifyHandler::WatchedDirectory::handle(struct inotify_event* event) {\n std::string basename;\n\n if (event->len > 0) {\n basename = event->name;\n }\n\n DEBUG_INFO << \"inotify event on: \" << path << \"\\n basename: \" << basename << \"\\n flags:\"\n << ((event->mask & IN_ATTRIB ) ? \" IN_ATTRIB\" : \"\")\n << ((event->mask & IN_CLOSE_WRITE) ? \" IN_CLOSE_WRITE\" : \"\")\n << ((event->mask & IN_CREATE ) ? \" IN_CREATE\" : \"\")\n << ((event->mask & IN_DELETE ) ? \" IN_DELETE\" : \"\")\n << ((event->mask & IN_DELETE_SELF) ? \" IN_DELETE_SELF\" : \"\")\n << ((event->mask & IN_MODIFY ) ? \" IN_MODIFY\" : \"\")\n << ((event->mask & IN_MOVE_SELF ) ? \" IN_MOVE_SELF\" : \"\")\n << ((event->mask & IN_MOVED_FROM ) ? \" IN_MOVED_FROM\" : \"\")\n << ((event->mask & IN_MOVED_TO ) ? \" IN_MOVED_TO\" : \"\");\n\n // Some events implicitly remove the watch descriptor (because the watched directory no longer\n // exists). Such descriptors are now invalid and may be reused the next time\n // inotify_add_watch() is called. But, the corresponding WatchedDirectories still have\n // FileWatcherImpls pointing at them, so we can't just delete them. So, invalidate them so\n // that no future FileWatcherImpls will use them.\n //\n // inotify has a special kind of event called IN_IGNORED which is supposed to signal that the\n // watch descriptor was removed, either implicitly or explicitly. However, in my testing,\n // this event does not seem to be generated in the case of IN_MOVE_SELF, even though this\n // case *does* implicitly remove the watch descriptor (which I know because the next call\n // to inotify_add_watch() typically reuses it). This appears to be a bug in Linux (observed\n // in 2.6.35-22-generic). Will file a bug report if I get time to write a demo program.\n if (event->mask & (IN_DELETE_SELF | IN_MOVE_SELF)) {\n DEBUG_INFO << \"Watch descriptor implicitly removed: \" << event->wd;\n invalidate();\n }\n\n for (CallbackTable::SearchIterator iter(callbackTable, basename);\n iter.next();) {\n FileWatcherImpl* op = iter.cell();\n if (event->mask & (IN_DELETE | IN_DELETE_SELF | IN_MOVED_FROM | IN_MOVE_SELF)) {\n op->flagAsDeleted();\n } else {\n op->flagAsModified();\n }\n }\n\n // If this event is indicating creation or deletion of a file in the directory, then call the\n // directory's modified() callback as well.\n if (!basename.empty() &&\n (event->mask & (IN_CREATE | IN_DELETE | IN_MOVED_FROM | IN_MOVED_TO))) {\n for (CallbackTable::SearchIterator iter(callbackTable, \"\");\n iter.next();) {\n FileWatcherImpl* op = iter.cell();\n op->flagAsModified();\n }\n }\n}\n\nvoid EpollEventManager::InotifyHandler::handle(uint32_t epollEvents) {\n char buffer[sizeof(struct inotify_event) + PATH_MAX];\n\n ssize_t n = inotifyStream.read(buffer, sizeof(buffer));\n\n char* pos = buffer;\n char* end = buffer + n;\n\n // Annoyingly, inotify() provides no way to read a single event at a time. If we were using\n // traditional callbacks for each event, then the callback for an earlier event could invalidate\n // later events. E.g. handling the first event might cause the watch descriptor for the second\n // event to be unregistered.\n //\n // As if that weren't bad enough, any particular event in the stream might indicate that a\n // particular watch descriptor has been automatically removed because the thing it was watching\n // no longer exists. This removal takes place at the time of the read(). So if the second\n // event in the buffer has the \"watch descriptor removed\" flag, and then while handling the\n // *first* event we create a new watch descriptor, that new descriptor may have the same\n // number as the one associated with the second event THAT WE HAVEN'T EVEN HANDLED YET.\n //\n // Luckily, when a promise is fulfilled, no callback is executed immediately; the callback\n // is queued on the event queue. Therefore, we don't have to worry about our caller coming\n // back and messing with our state while we're still going through the event list.\n\n while (pos < end) {\n if (end - pos < (signed)sizeof(struct inotify_event)) {\n DEBUG_ERROR << \"read(inotifyFd) returned too few bytes to be an inotify_event.\";\n break;\n }\n\n struct inotify_event* event = reinterpret_cast(pos);\n\n if (end - pos - sizeof(struct inotify_event) < event->len) {\n DEBUG_ERROR\n << \"read(inotifyFd) returned inotify_event with 'len' that overruns the buffer.\";\n break;\n }\n\n pos += sizeof(struct inotify_event) + event->len;\n\n DEBUG_INFO << \"inotify \" << event->wd << \":\"\n << ((event->mask & IN_ATTRIB ) ? \" IN_ATTRIB\" : \"\")\n << ((event->mask & IN_CLOSE_WRITE) ? \" IN_CLOSE_WRITE\" : \"\")\n << ((event->mask & IN_CREATE ) ? \" IN_CREATE\" : \"\")\n << ((event->mask & IN_DELETE ) ? \" IN_DELETE\" : \"\")\n << ((event->mask & IN_DELETE_SELF) ? \" IN_DELETE_SELF\" : \"\")\n << ((event->mask & IN_MODIFY ) ? \" IN_MODIFY\" : \"\")\n << ((event->mask & IN_MOVE_SELF ) ? \" IN_MOVE_SELF\" : \"\")\n << ((event->mask & IN_MOVED_FROM ) ? \" IN_MOVED_FROM\" : \"\")\n << ((event->mask & IN_MOVED_TO ) ? \" IN_MOVED_TO\" : \"\")\n << ((event->mask & IN_IGNORED ) ? \" IN_IGNORED\" : \"\");\n\n WatchMap::iterator iter = watchMap.find(event->wd);\n if (iter == watchMap.end()) {\n if (event->mask != IN_IGNORED) {\n DEBUG_ERROR << \"inotify event had unknown watch descriptor? \" << event->wd;\n }\n } else {\n iter->second->handle(event);\n }\n }\n}\n\nOwnedPtr EpollEventManager::InotifyHandler::watchFile(\n const std::string& filename) {\n return newOwned(this, filename);\n}\n\nOwnedPtr EpollEventManager::watchFile(const std::string& filename) {\n return inotifyHandler.watchFile(filename);\n}\n\n// =======================================================================================\n\nEpollEventManager::EpollEventManager()\n : signalHandler(&epoller), inotifyHandler(&epoller) {}\nEpollEventManager::~EpollEventManager() {}\n\nvoid EpollEventManager::loop() {\n while (handleEvent()) {}\n}\n\nbool EpollEventManager::handleEvent() {\n // Run any async callbacks first.\n // TODO: Avoid starvation of I/O? Probably doesn't matter.\n if (!asyncCallbacks.empty()) {\n AsyncCallbackHandler* handler = asyncCallbacks.front();\n asyncCallbacks.pop_front();\n handler->run();\n return true;\n }\n\n return epoller.handleEvent();\n}\n\n// =======================================================================================\n\nOwnedPtr newPreferredEventManager() {\n return newOwned();\n}\n\n} // namespace ekam\n" }, { "path": "src/os/EpollEventManager.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_OS_EPOLLEVENTMANAGER_H_\n#define KENTONSCODE_OS_EPOLLEVENTMANAGER_H_\n\n#include \n#include \n#include \n#include \n#include \n#include \n#include \n\n#include \"EventManager.h\"\n#include \"base/OwnedPtr.h\"\n#include \"OsHandle.h\"\n#include \"ByteStream.h\"\n\ntypedef struct pollfd PollFd;\n\nnamespace ekam {\n\nclass EpollEventManager : public RunnableEventManager {\npublic:\n EpollEventManager();\n ~EpollEventManager();\n\n // implements RunnableEventManager -----------------------------------------------------\n void loop();\n\n // implements Executor -----------------------------------------------------------------\n OwnedPtr runLater(OwnedPtr runnable);\n\n // implements EventManager -------------------------------------------------------------\n Promise onProcessExit(pid_t pid);\n OwnedPtr watchFd(int fd);\n OwnedPtr watchFile(const std::string& filename);\n\nprivate:\n class AsyncCallbackHandler;\n class IoWatcherImpl;\n\n class IoHandler {\n public:\n virtual ~IoHandler() noexcept(false) {}\n\n virtual void handle(uint32_t events) = 0;\n };\n\n class Epoller {\n public:\n Epoller();\n ~Epoller();\n\n bool handleEvent();\n\n class Watch {\n public:\n Watch(Epoller* epoller, OsHandle* handle, uint32_t events, IoHandler* handler);\n Watch(Epoller* epoller, int fd, uint32_t events, IoHandler* handler);\n ~Watch();\n\n // Add or remove events being watched.\n void addEvents(uint32_t eventsToAdd);\n void removeEvents(uint32_t eventsToRemove);\n\n private:\n friend class Epoller;\n\n Epoller* epoller;\n uint32_t events;\n uint32_t registeredEvents;\n int fd;\n std::string name;\n IoHandler* handler;\n\n void updateRegistration();\n };\n\n private:\n OsHandle epollHandle;\n int watchCount;\n\n std::unordered_set watchesNeedingUpdate;\n };\n\n class SignalHandler : public IoHandler {\n public:\n SignalHandler(Epoller* epoller);\n ~SignalHandler();\n\n Promise onProcessExit(pid_t pid);\n\n // implements IoHandler --------------------------------------------------------------\n void handle(uint32_t events);\n\n private:\n class ProcessExitHandler;\n\n ByteStream signalStream;\n Epoller::Watch watch;\n std::unordered_map processExitHandlerMap;\n\n void handleProcessExit();\n void maybeStopExpecting();\n };\n\n class InotifyHandler : public IoHandler {\n public:\n InotifyHandler(Epoller* epoller);\n ~InotifyHandler();\n\n OwnedPtr watchFile(const std::string& filename);\n\n // implements IoHandler --------------------------------------------------------------\n void handle(uint32_t events);\n\n private:\n class WatchedDirectory;\n class FileWatcherImpl;\n\n ByteStream inotifyStream;\n Epoller::Watch watch;\n\n OwnedPtrMap ownedWatchDirectories;\n typedef std::unordered_map WatchMap;\n WatchMap watchMap;\n typedef std::unordered_map WatchByNameMap;\n WatchByNameMap watchByNameMap;\n };\n\n Epoller epoller;\n SignalHandler signalHandler;\n InotifyHandler inotifyHandler;\n\n std::deque asyncCallbacks;\n\n bool handleEvent();\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_OS_EPOLLEVENTMANAGER_H_\n" }, { "path": "src/os/EventGroup.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"EventGroup.h\"\n\n#include \"base/Debug.h\"\n\nnamespace ekam {\n\nEventGroup::ExceptionHandler::~ExceptionHandler() noexcept(false) {}\n\nclass EventGroup::PendingEvent {\npublic:\n PendingEvent(EventGroup* group): group(group) {\n ++group->eventCount;\n }\n ~PendingEvent() {\n if (--group->eventCount == 0) {\n group->callNoMoreEventsLater();\n }\n }\n\nprivate:\n EventGroup* group;\n};\n\n\n#define HANDLE_EXCEPTIONS(STATEMENT) \\\n EventGroup* group = this->group; \\\n try { \\\n STATEMENT; \\\n } catch (const std::exception& exception) { \\\n group->exceptionHandler->threwException(exception); \\\n } catch (...) { \\\n group->exceptionHandler->threwUnknownException(); \\\n }\n\nclass EventGroup::RunnableWrapper : public Runnable {\npublic:\n RunnableWrapper(EventGroup* group, OwnedPtr wrapped)\n : group(group), pendingEvent(group), wrapped(wrapped.release()) {}\n ~RunnableWrapper() {}\n\n // implements Runnable -----------------------------------------------------------------\n void run() { HANDLE_EXCEPTIONS(wrapped->run()); }\n\nprivate:\n EventGroup* group;\n PendingEvent pendingEvent;\n OwnedPtr wrapped;\n};\n\n#undef HANDLE_EXCEPTIONS\n\n// =======================================================================================\n\nEventGroup::EventGroup(EventManager* inner, ExceptionHandler* exceptionHandler)\n : inner(inner), exceptionHandler(exceptionHandler), eventCount(0) {}\n\nEventGroup::~EventGroup() {}\n\nOwnedPtr EventGroup::runLater(OwnedPtr runnable) {\n auto wrappedCallback = newOwned(this, runnable.release());\n return inner->runLater(wrappedCallback.release());\n}\n\nPromise EventGroup::onProcessExit(pid_t pid) {\n Promise innerPromise = inner->onProcessExit(pid);\n return when(innerPromise, newPendingEvent())(\n [](ProcessExitCode exitCode, OwnedPtr) -> ProcessExitCode {\n return exitCode;\n });\n}\n\nclass EventGroup::IoWatcherWrapper: public EventManager::IoWatcher {\npublic:\n IoWatcherWrapper(EventGroup* group, OwnedPtr inner)\n : group(group), inner(inner.release()) {}\n ~IoWatcherWrapper() {}\n\n // implements IoWatcher ----------------------------------------------------------------\n Promise onReadable() {\n Promise innerPromise = inner->onReadable();\n return group->when(innerPromise, group->newPendingEvent())(\n [](Void, OwnedPtr) {\n // Let PendingEvent die.\n });\n }\n Promise onWritable() {\n Promise innerPromise = inner->onWritable();\n return group->when(innerPromise, group->newPendingEvent())(\n [](Void, OwnedPtr) {\n // Let PendingEvent die.\n });\n }\n\nprivate:\n EventGroup* group;\n OwnedPtr inner;\n};\n\nOwnedPtr EventGroup::watchFd(int fd) {\n return newOwned(this, inner->watchFd(fd));\n}\n\nclass EventGroup::FileWatcherWrapper: public EventManager::FileWatcher {\npublic:\n FileWatcherWrapper(EventGroup* group, OwnedPtr inner)\n : group(group), inner(inner.release()) {}\n ~FileWatcherWrapper() {}\n\n // implements IoWatcher ----------------------------------------------------------------\n Promise onChange() {\n Promise innerPromise = inner->onChange();\n return group->when(innerPromise, group->newPendingEvent())(\n [](FileChangeType changeType, OwnedPtr) -> FileChangeType {\n // Let PendingEvent die.\n return changeType;\n });\n }\n\nprivate:\n EventGroup* group;\n OwnedPtr inner;\n};\n\nOwnedPtr EventGroup::watchFile(const std::string& filename) {\n return newOwned(this, inner->watchFile(filename));\n}\n\nOwnedPtr EventGroup::newPendingEvent() {\n return newOwned(this);\n}\n\nvoid EventGroup::callNoMoreEventsLater() {\n pendingNoMoreEvents = inner->when()(\n [this]() {\n pendingNoMoreEvents.release();\n if (eventCount == 0) {\n DEBUG_INFO << \"No more events on EventGroup.\";\n exceptionHandler->noMoreEvents();\n }\n });\n}\n\n} // namespace ekam\n" }, { "path": "src/os/EventGroup.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_OS_EVENTGROUP_H_\n#define KENTONSCODE_OS_EVENTGROUP_H_\n\n#include \n\n#include \"EventManager.h\"\n\nnamespace ekam {\n\n// A wrapper around an EventManager which keeps track of all the events being waited on\n// through it and calls a callback when there is nothing left to do. Additionally, exceptions\n// thrown by callbacks are caught and reported.\n//\n// TODO: Better name?\nclass EventGroup: public EventManager {\npublic:\n class ExceptionHandler {\n public:\n virtual ~ExceptionHandler() noexcept(false);\n\n // An event callback threw an exception. The ExceptionHandler is expected to immediately\n // cancel the event group such that no further events are received by it.\n virtual void threwException(const std::exception& e) = 0;\n virtual void threwUnknownException() = 0;\n\n // Indicates that this group completed successfully -- it is no longer waiting for anything\n // and no exception was thrown.\n virtual void noMoreEvents() = 0;\n };\n\n EventGroup(EventManager* inner, ExceptionHandler* exceptionHandler);\n ~EventGroup();\n\n // implements Executor -----------------------------------------------------------------\n OwnedPtr runLater(OwnedPtr runnable);\n\n // implements EventManager -------------------------------------------------------------\n Promise onProcessExit(pid_t pid);\n OwnedPtr watchFd(int fd);\n OwnedPtr watchFile(const std::string& filename);\n\nprivate:\n class PendingEvent;\n\n class RunnableWrapper;\n class IoWatcherWrapper;\n class FileWatcherWrapper;\n\n EventManager* inner;\n ExceptionHandler* exceptionHandler;\n int eventCount;\n Promise pendingNoMoreEvents;\n\n OwnedPtr newPendingEvent();\n void callNoMoreEventsLater();\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_OS_EVENTGROUP_H_\n" }, { "path": "src/os/EventManager.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"EventManager.h\"\n\n#include \n\n#include \"OsHandle.h\" // temporary, for toString()\n\nnamespace ekam {\n\nEventManager::~EventManager() noexcept(false) {}\nEventManager::IoWatcher::~IoWatcher() noexcept(false) {}\nEventManager::FileWatcher::~FileWatcher() {}\nRunnableEventManager::~RunnableEventManager() noexcept(false) {}\n\nvoid ProcessExitCode::throwError() {\n if (signaled) {\n throw std::logic_error(\"Process was signaled: \" + toString(exitCodeOrSignal));\n } else {\n throw std::logic_error(\"Process was not signaled. Exit code: \" + toString(exitCodeOrSignal));\n }\n}\n\n} // namespace ekam\n" }, { "path": "src/os/EventManager.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_OS_EVENTMANAGER_H_\n#define KENTONSCODE_OS_EVENTMANAGER_H_\n\n#include \n#include \n#include \n#include \"base/OwnedPtr.h\"\n#include \"base/Promise.h\"\n\nnamespace ekam {\n\nclass ProcessExitCode {\npublic:\n ProcessExitCode(): signaled(false), exitCodeOrSignal(0) {}\n ProcessExitCode(int exitCode)\n : signaled(false), exitCodeOrSignal(exitCode) {}\n enum Signaled { SIGNALED };\n ProcessExitCode(Signaled, int signalNumber)\n : signaled(true), exitCodeOrSignal(signalNumber) {}\n\n bool wasSignaled() {\n return signaled;\n }\n\n int getExitCode() {\n if (signaled) {\n throwError();\n }\n return exitCodeOrSignal;\n }\n\n int getSignalNumber() {\n if (!signaled) {\n throwError();\n }\n return exitCodeOrSignal;\n }\n\nprivate:\n bool signaled;\n int exitCodeOrSignal;\n\n void throwError();\n};\n\nclass EventManager : public Executor {\npublic:\n virtual ~EventManager() noexcept(false);\n\n // Fulfills the promise when the process exits.\n virtual Promise onProcessExit(pid_t pid) = 0;\n\n class IoWatcher {\n public:\n virtual ~IoWatcher() noexcept(false);\n\n // Fulfills the promise when the file descriptor is readable.\n virtual Promise onReadable() = 0;\n // Fulfills the promise when the file descriptor is writable.\n virtual Promise onWritable() = 0;\n };\n\n // Watch the file descriptor for readability and writability.\n virtual OwnedPtr watchFd(int fd) = 0;\n\n enum class FileChangeType {\n MODIFIED,\n DELETED\n };\n\n class FileWatcher {\n public:\n virtual ~FileWatcher();\n\n virtual Promise onChange() = 0;\n };\n\n // Watch a file (on disk) for changes or deletion.\n virtual OwnedPtr watchFile(const std::string& filename) = 0;\n};\n\nclass RunnableEventManager : public EventManager {\npublic:\n virtual ~RunnableEventManager() noexcept(false);\n\n virtual void loop() = 0;\n};\n\nOwnedPtr newPreferredEventManager();\n\n} // namespace ekam\n\n#endif // KENTONSCODE_OS_EVENTMANAGER_H_\n" }, { "path": "src/os/File.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"File.h\"\n#include \n\nnamespace ekam {\n\nFile::~File() {}\nFile::DiskRef::~DiskRef() {};\n\nvoid splitExtension(const std::string& name, std::string* base, std::string* ext) {\n std::string::size_type pos = name.find_last_of('.');\n std::string::size_type slashpos = name.find_last_of('/');\n if (pos == std::string::npos || (slashpos != std::string::npos && pos < slashpos)) {\n base->assign(name);\n ext->clear();\n } else {\n base->assign(name, 0, pos);\n ext->assign(name, pos, std::string::npos);\n }\n}\n\nvoid recursivelyCreateDirectory(File* location) {\n if (!location->isDirectory()) {\n recursivelyCreateDirectory(location->parent().get());\n location->createDirectory();\n }\n}\n\n} // namespace ekam\n" }, { "path": "src/os/File.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_OS_FILE_H_\n#define KENTONSCODE_OS_FILE_H_\n\n#include \n#include \n\n#include \"base/OwnedPtr.h\"\n\nnamespace ekam {\n\nclass Hash;\n\nclass File {\npublic:\n virtual ~File();\n\n virtual std::string basename() = 0;\n virtual std::string canonicalName() = 0;\n virtual OwnedPtr clone() = 0;\n virtual bool hasParent() = 0;\n virtual OwnedPtr parent() = 0;\n\n virtual bool equals(File* other) = 0;\n virtual size_t identityHash() = 0;\n\n class HashFunc {\n public:\n inline size_t operator()(File* file) const {\n return file->identityHash();\n }\n };\n struct EqualFunc {\n inline bool operator()(File* a, File* b) const {\n return a->equals(b);\n }\n };\n\n class DiskRef {\n public:\n virtual ~DiskRef();\n\n virtual const std::string& path() = 0;\n };\n\n enum Usage {\n READ,\n WRITE,\n UPDATE\n };\n\n virtual OwnedPtr getOnDisk(Usage usage) = 0;\n\n virtual bool exists() = 0;\n virtual bool isFile() = 0;\n virtual bool isDirectory() = 0;\n\n // File only.\n virtual Hash contentHash() = 0;\n virtual std::string readAll() = 0;\n virtual void writeAll(const std::string& content) = 0;\n virtual void writeAll(const void* data, int size) = 0;\n\n // Directory only.\n virtual void list(OwnedPtrVector::Appender output) = 0;\n virtual OwnedPtr relative(const std::string& path) = 0;\n\n // Methods that create or delete objects.\n virtual void createDirectory() = 0;\n virtual void link(File* target) = 0;\n virtual void unlink() = 0;\n};\n\nvoid splitExtension(const std::string& name, std::string* base, std::string* ext);\nvoid recursivelyCreateDirectory(File* location);\n\n} // namespace ekam\n\n#endif // KENTONSCODE_OS_FILE_H_\n" }, { "path": "src/os/KqueueEventManager.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_OS_KQUEUEEVENTMANAGER_H_\n#define KENTONSCODE_OS_KQUEUEEVENTMANAGER_H_\n\n#include \n#include \n#include \n\n#include \"EventManager.h\"\n#include \"base/OwnedPtr.h\"\n\ntypedef struct kevent KEvent;\n\nnamespace ekam {\n\nclass KqueueEventManager: public RunnableEventManager {\npublic:\n KqueueEventManager();\n ~KqueueEventManager();\n\n // implements RunnableEventManager -----------------------------------------------------\n void loop();\n\n // implements EventManager -------------------------------------------------------------\n OwnedPtr runAsynchronously(Callback* callback);\n OwnedPtr onProcessExit(pid_t pid, ProcessExitCallback* callback);\n OwnedPtr onReadable(int fd, IoCallback* callback);\n OwnedPtr onWritable(int fd, IoCallback* callback);\n OwnedPtr onFileChange(const std::string& filename, FileChangeCallback* callback);\n\nprivate:\n class KEventHandler;\n\n class AsyncCallbackHandler;\n class ProcessExitHandler;\n class ReadHandler;\n class WriteHandler;\n class FileChangeHandler;\n\n struct IntptrShortPairHash {\n inline bool operator()(const std::pair& p) const {\n return p.first * 65537 + p.second;\n }\n };\n\n int kqueueFd;\n\n std::deque asyncCallbacks;\n std::deque fakeEvents;\n int handlerCount;\n\n bool handleEvent();\n\n void updateKqueue(const KEvent& event);\n void updateKqueue(uintptr_t ident, short filter, u_short flags,\n KEventHandler* handler = NULL, u_int fflags = 0, intptr_t data = 0);\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_OS_KQUEUEEVENTMANAGER_H_\n" }, { "path": "src/os/OsHandle.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"OsHandle.h\"\n\n#include \n#include \n#include \n#include \n#include \n#include \n#include \n\n#include \"base/Debug.h\"\n\nnamespace ekam {\n\nOsHandle::OsHandle(const std::string& name, int fd)\n : name(name), fd(fd) {\n if (fd < 0) {\n throw std::invalid_argument(\"Negative file descriptor given for: \" + name);\n }\n fcntl(fd, F_SETFD, FD_CLOEXEC);\n}\n\nOsHandle::~OsHandle() {\n if (close(fd) < 0) {\n DEBUG_ERROR << \"close(\" << name << \"): \" << strerror(errno);\n }\n}\n\nstd::string toString(const char* arg) {\n return arg;\n}\nstd::string toString(int arg) {\n std::stringstream sout(std::stringstream::out);\n sout << arg;\n return sout.str();\n}\nstd::string toString(const OsHandle& arg) {\n return arg.getName();\n}\n\nOsError::OsError(const std::string& path, const char* function, int errorNumber)\n : errorNumber(errorNumber) {\n if (function != NULL && *function != '\\0') {\n description.append(function);\n if (!path.empty()) {\n description.append(\"(\");\n description.append(path);\n description.append(\")\");\n }\n description.append(\": \");\n } else if (!path.empty()) {\n description.append(path);\n description.append(\": \");\n }\n description.append(strerror(errorNumber));\n}\n\nOsError::OsError(const char* function, int errorNumber)\n : errorNumber(errorNumber) {\n if (function != NULL && *function != '\\0') {\n description.append(function);\n description.append(\": \");\n }\n description.append(strerror(errorNumber));\n}\n\nOsError::~OsError() throw() {}\n\nconst char* OsError::what() const throw() {\n return description.c_str();\n}\n\n} // namespace ekam\n" }, { "path": "src/os/OsHandle.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_OS_OSHANDLE_H_\n#define KENTONSCODE_OS_OSHANDLE_H_\n\n#include \n#include \n#include \n\n#include \"base/OwnedPtr.h\"\n\nnamespace ekam {\n\nclass EventManager;\n\nclass OsHandle {\npublic:\n OsHandle(const std::string& name, int fd);\n ~OsHandle();\n\n const std::string& getName() const { return name; }\n int get() const { return fd; }\n\nprivate:\n const std::string name;\n const int fd;\n};\n\nclass OsError : public std::exception {\npublic:\n OsError(const std::string& path, const char* function, int errorNumber);\n OsError(const char* function, int errorNumber);\n virtual ~OsError() throw();\n\n virtual const char* what() const throw();\n\n inline int getErrorNumber() const { return errorNumber; }\n\nprivate:\n std::string description;\n int errorNumber;\n};\n\n// TODO(kenton): Factor out toString() module.\nstd::string toString(const char* arg);\nstd::string toString(int arg);\nstd::string toString(const OsHandle& arg);\n\ntemplate \ninline const T& toSyscallArg(const T& value) { return value; }\ninline int toSyscallArg(const OsHandle& value) { return value.get(); }\n\ntemplate \nlong wrapSyscall(const char* name, const Func& func) {\n long result;\n do {\n result = func();\n } while (result < 0 && errno == EINTR);\n if (result < 0) {\n throw OsError(name, errno);\n }\n return result;\n}\n\ntemplate \nlong wrapSyscall(const char* name, const Func& func, Arg1&& arg1, Args&&... args) {\n long result;\n do {\n result = func(toSyscallArg(arg1), toSyscallArg(args)...);\n } while (result < 0 && errno == EINTR);\n if (result < 0) {\n throw OsError(toString(arg1), name, errno);\n }\n return result;\n}\n\n// The ## tells GCC to omit the preceding comma if __VA_ARGS__ is empty. This is non-standard,\n// but apparently MSVC will do the same.\n#define WRAP_SYSCALL(FUNC, ...) (::ekam::wrapSyscall(#FUNC, ::FUNC, ##__VA_ARGS__))\n\n} // namespace ekam\n\n#endif // KENTONSCODE_OS_OSHANDLE_H_\n" }, { "path": "src/os/PollEventManager.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_OS_POLLEVENTMANAGER_H_\n#define KENTONSCODE_OS_POLLEVENTMANAGER_H_\n\n#include \n#include \n#include \n#include \n#include \n\n#include \"EventManager.h\"\n#include \"base/OwnedPtr.h\"\n\ntypedef struct pollfd PollFd;\n\nnamespace ekam {\n\nclass PollEventManager : public RunnableEventManager {\npublic:\n PollEventManager();\n ~PollEventManager();\n\n // implements RunnableEventManager -----------------------------------------------------\n void loop();\n\n // implements EventManager -------------------------------------------------------------\n OwnedPtr runAsynchronously(Callback* callback);\n OwnedPtr onProcessExit(pid_t pid, ProcessExitCallback* callback);\n OwnedPtr onReadable(int fd, IoCallback* callback);\n OwnedPtr onWritable(int fd, IoCallback* callback);\n OwnedPtr onFileChange(const std::string& filename, FileChangeCallback* callback);\n\nprivate:\n class IoHandler;\n\n class AsyncCallbackHandler;\n class ProcessExitHandler;\n class ReadHandler;\n class WriteHandler;\n\n std::deque asyncCallbacks;\n std::unordered_map processExitHandlerMap;\n std::unordered_map readHandlerMap;\n std::unordered_map writeHandlerMap;\n\n bool handleEvent();\n void handleSignal(const siginfo_t& siginfo);\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_OS_POLLEVENTMANAGER_H_\n" }, { "path": "src/os/Socket.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"Socket.h\"\n\n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n\n#include \"base/Debug.h\"\n\nnamespace ekam {\n\nnamespace {\n\nstd::string splitFirst(std::string* str, char delim) {\n std::string::size_type pos = str->find_first_of(delim);\n std::string result;\n if (pos == std::string::npos) {\n result = *str;\n str->clear();\n } else {\n result.assign(*str, 0, pos);\n str->erase(0, pos + 1);\n }\n return result;\n}\n\nbool parseIpAddr(std::string text, struct sockaddr_in* addr) {\n // TODO: This code sucks. Find a library to call.\n\n addr->sin_family = AF_INET;\n\n std::string address = splitFirst(&text, ':');\n if (text.empty()) return false;\n\n std::vector parts;\n while (!address.empty()) {\n std::string part = splitFirst(&address, '.');\n char* end;\n parts.push_back(strtoul(part.c_str(), &end, 0));\n if (end != part.data() + part.size()) return false;\n }\n\n if (parts.size() > 4) return false;\n\n addr->sin_addr.s_addr = 0;\n\n if (!parts.empty()) {\n for (size_t i = 0; i < parts.size() - 1; i++) {\n if (parts[i] > 0xFFu) return false;\n addr->sin_addr.s_addr |= parts[i] << ((3 - i) * 8);\n }\n\n if (parts.back() > (0xFFFFFFFFu >> ((parts.size() - 1) * 8))) return false;\n addr->sin_addr.s_addr |= parts.back();\n\n addr->sin_addr.s_addr = htonl(addr->sin_addr.s_addr);\n }\n\n char* end;\n unsigned long port = strtoul(text.c_str(), &end, 0);\n if (end != text.data() + text.size() || port > 0xFFFFu) return false;\n addr->sin_port = htons(port);\n\n return true;\n}\n\n} // namespace\n\nServerSocket::ServerSocket(EventManager* eventManager, const std::string& bindAddress, int backlog)\n : eventManager(eventManager),\n handle(bindAddress, WRAP_SYSCALL(socket, AF_INET, SOCK_STREAM, 0)),\n watcher(eventManager->watchFd(handle.get())) {\n WRAP_SYSCALL(fcntl, handle, F_SETFL, O_NONBLOCK);\n\n int optval = 1;\n WRAP_SYSCALL(setsockopt, handle, SOL_SOCKET, SO_REUSEADDR, &optval, sizeof(optval));\n\n struct sockaddr_in addr;\n if (!parseIpAddr(bindAddress, &addr)) {\n throw std::invalid_argument(\"Invalid bind address: \" + bindAddress);\n }\n\n WRAP_SYSCALL(bind, handle, reinterpret_cast(&addr), sizeof(addr));\n WRAP_SYSCALL(listen, handle, (backlog == 0) ? SOMAXCONN : backlog);\n}\n\nServerSocket::~ServerSocket() {}\n\nPromise> ServerSocket::accept() {\n return eventManager->when(watcher->onReadable())(\n [this](Void) -> Promise> {\n int fd = ::accept(handle.get(), NULL, NULL);\n if (fd < 0) {\n switch (errno) {\n case EINTR:\n case ECONNABORTED:\n case EAGAIN:\n#if EAGAIN != EWOULDBLOCK\n case EWOULDBLOCK:\n#endif\n // This are \"normal\". Try again.\n DEBUG_INFO << \"accept: \" << strerror(errno);\n return accept();\n default:\n throw OsError(\"accept\", errno);\n break;\n }\n } else {\n // TODO: Use peer address as name.\n return newFulfilledPromise(newOwned(fd, \"accepted connection\"));\n }\n });\n}\n\n\n} // namespace ekam\n" }, { "path": "src/os/Socket.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_OS_SOCKET_H_\n#define KENTONSCODE_OS_SOCKET_H_\n\n#include \"base/OwnedPtr.h\"\n#include \"OsHandle.h\"\n#include \"ByteStream.h\"\n#include \"EventManager.h\"\n\nnamespace ekam {\n\nclass ServerSocket {\npublic:\n ServerSocket(EventManager* eventManager, const std::string& bindAddress, int backlog = 0);\n ~ServerSocket();\n\n Promise> accept();\n\nprivate:\n class AcceptOp;\n\n EventManager* eventManager;\n OsHandle handle;\n OwnedPtr watcher;\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_OS_SOCKET_H_\n" }, { "path": "src/os/Subprocess.cpp", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#include \"Subprocess.h\"\n\n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n\n#include \"OsHandle.h\"\n#include \"base/Debug.h\"\n\nnamespace ekam {\n\nSubprocess::Subprocess() : doPathLookup(false), pid(-1) {}\n\nSubprocess::~Subprocess() {\n if (pid >= 0) {\n DEBUG_INFO << \"Killing pid: \" << pid;\n // Kill entire progress group.\n kill(-pid, SIGKILL);\n int dummy;\n waitpid(pid, &dummy, 0);\n }\n}\n\nvoid Subprocess::addArgument(const std::string& arg) {\n if (args.empty()) {\n executableName = arg;\n doPathLookup = true;\n }\n args.push_back(arg);\n}\n\nFile::DiskRef* Subprocess::addArgument(File* file, File::Usage usage) {\n OwnedPtr diskRef = file->getOnDisk(usage);\n\n if (args.empty()) {\n executableName = diskRef->path();\n doPathLookup = false;\n }\n args.push_back(diskRef->path());\n\n File::DiskRef* result = diskRef.get();\n diskRefs.add(diskRef.release());\n return result;\n}\n\nOwnedPtr Subprocess::captureStdin() {\n stdinPipe = newOwned();\n return stdinPipe->releaseWriteEnd();\n}\n\nOwnedPtr Subprocess::captureStdout() {\n stdoutPipe = newOwned();\n stdoutAndStderrPipe.clear();\n return stdoutPipe->releaseReadEnd();\n}\n\nOwnedPtr Subprocess::captureStderr() {\n stderrPipe = newOwned();\n stdoutAndStderrPipe.clear();\n return stderrPipe->releaseReadEnd();\n}\n\nOwnedPtr Subprocess::captureStdoutAndStderr() {\n stdoutAndStderrPipe = newOwned();\n stdoutPipe.clear();\n stderrPipe.clear();\n return stdoutAndStderrPipe->releaseReadEnd();\n}\n\nPromise Subprocess::start(EventManager* eventManager) {\n pid = fork();\n\n if (pid < 0) {\n throw OsError(\"\", \"fork\", errno);\n } else if (pid == 0) {\n // In child.\n\n std::vector argv;\n std::string command;\n\n for (unsigned int i = 0; i < args.size(); i++) {\n argv.push_back(strdup(args[i].c_str()));\n\n if (i > 0) command.push_back(' ');\n command.append(args[i]);\n }\n\n argv.push_back(NULL);\n\n DEBUG_INFO << \"exec: \" << command;\n\n if (stdinPipe != NULL) {\n stdinPipe->attachReadEndForExec(STDIN_FILENO);\n }\n if (stdoutPipe != NULL) {\n stdoutPipe->attachWriteEndForExec(STDOUT_FILENO);\n }\n if (stderrPipe != NULL) {\n stderrPipe->attachWriteEndForExec(STDERR_FILENO);\n }\n if (stdoutAndStderrPipe != NULL) {\n stdoutAndStderrPipe->attachWriteEndForExec(STDOUT_FILENO);\n dup2(STDOUT_FILENO, STDERR_FILENO);\n }\n\n // Start a new progress group so that we can kill it all at once.\n // TODO(someday): This means if you ctrl+C ekam itself, the SIGINT is not distributed to jobs\n // running under it. Can we fix that? Another thing we could do is put the job into a PID\n // namespace but that's a lot more work and requires user namespaces and only works on Linux.\n // Probably what we have to do is handle sigint ourselves and redistribute it to all\n // children, bleh.\n setpgid(0, 0);\n\n if (doPathLookup) {\n execvp(executableName.c_str(), &argv[0]);\n } else {\n execv(executableName.c_str(), &argv[0]);\n }\n\n perror(\"exec\");\n exit(1);\n } else {\n if (stdoutPipe != NULL) {\n stdoutPipe.clear();\n }\n if (stdinPipe != NULL) {\n stdinPipe.clear();\n }\n if (stderrPipe != NULL) {\n stderrPipe.clear();\n }\n if (stdoutAndStderrPipe != NULL) {\n stdoutAndStderrPipe.clear();\n }\n\n // Set the child's process group ID. The child also does this to itself (see above), but we\n // need to do it in the parent as well to prevent a race condition in which we end up killing\n // the child before it manages to call setpgid(). If that happens, then the child will keep\n // going and the parent process will end up blockend on waitpid(). But the child process will\n // almost certainly make an RPC to the parent process and wait for a reply, leading to\n // deadlock.\n setpgid(pid, 0);\n\n return eventManager->when(eventManager->onProcessExit(pid))(\n [this](ProcessExitCode exitCode) -> ProcessExitCode {\n pid = -1;\n return exitCode;\n });\n }\n}\n\n} // namespace ekam\n" }, { "path": "src/os/Subprocess.h", "content": "// Ekam Build System\n// Author: Kenton Varda (kenton@sandstorm.io)\n// Copyright (c) 2010-2015 Kenton Varda, Google Inc., and contributors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n#ifndef KENTONSCODE_OS_SUBPROCESS_H_\n#define KENTONSCODE_OS_SUBPROCESS_H_\n\n#include \n#include \n\n#include \"base/OwnedPtr.h\"\n#include \"EventManager.h\"\n#include \"ByteStream.h\"\n#include \"File.h\"\n\nnamespace ekam {\n\nclass Subprocess {\npublic:\n Subprocess();\n ~Subprocess();\n\n void addArgument(const std::string& arg);\n File::DiskRef* addArgument(File* file, File::Usage usage);\n\n OwnedPtr captureStdin();\n OwnedPtr captureStdout();\n OwnedPtr captureStderr();\n OwnedPtr captureStdoutAndStderr();\n\n Promise start(EventManager* eventManager);\n\nprivate:\n class CallbackWrapper;\n\n std::string executableName;\n bool doPathLookup;\n\n std::vector args;\n OwnedPtrVector diskRefs;\n\n OwnedPtr stdinPipe;\n OwnedPtr stdoutPipe;\n OwnedPtr stderrPipe;\n OwnedPtr stdoutAndStderrPipe;\n\n pid_t pid;\n};\n\n} // namespace ekam\n\n#endif // KENTONSCODE_OS_SUBPROCESS_H_\n" }, { "path": "vscode/.gitignore", "content": "out\nnode_modules\npackage-lock.json\n*.vsix\n" }, { "path": "vscode/LICENSE", "content": "The MIT License (MIT)\n\nCopyright (c) 2018 Kenton Varda\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "vscode/README.md", "content": "# Ekam VS Code Plugin\n\nBrings error reports from Ekam into Visual Studio Code.\n\n## Usage\n\nThe extension will look for `ekam-langserve` in your `PATH`, or you can specify\nits location in your config like:\n\n```json\n{\n \"ekam.path\": \"/absolute/path/to/ekam-langserve\",\n \"ekam.args\": [ \"localhost:41315\" ]\n}\n```\n\nTo obtain `ekam-langserve` binary, build [Ekam](https://github.com/capnproto/ekam).\nThe binary will end up in `bin/ekam-langserve` after the build completes.\n\nThe language server expects to connect to a local Ekam run. You'll need to tell\nEkam to publish logs on a local port by running it like:\n\n ekam -c -n :41315\n\n## Building from source\n\n```bash\nnpm install\nnpm run postinstall\nnpm run package\n```\n\nThis builds `vscode-ekam.vsix`, which you can then install into VS Code.\n" }, { "path": "vscode/package.json", "content": "{\n \"name\": \"vscode-ekam\",\n \"displayName\": \"vscode-ekam\",\n \"description\": \"Ekam Language Server\",\n \"version\": \"0.2.0\",\n \"publisher\": \"kentonv\",\n \"homepage\": \"https://github.com/sandstorm-io/ekam\",\n \"engines\": {\n \"vscode\": \"^1.27.0\"\n },\n \"keywords\": [\n \"LSP\",\n \"Ekam\"\n ],\n \"activationEvents\": [\n \"onLanguage:cpp\"\n ],\n \"main\": \"./out/src/extension\",\n \"scripts\": {\n \"vscode:prepublish\": \"tsc -p ./\",\n \"compile\": \"tsc -watch -p ./\",\n \"package\": \"vsce package\"\n },\n \"dependencies\": {\n \"vscode-languageclient\": \"6.x\",\n \"vscode-languageserver\": \"6.x\"\n },\n \"devDependencies\": {\n \"typescript\": \"^4.9.4\",\n \"@types/vscode\": \"^1.1.0\",\n \"@types/node\": \"^6.0.40\",\n \"vsce\": \"^1.51.0\"\n },\n \"repository\": {\n \"type\": \"git\",\n \"url\": \"http://github.com/sandstorm-io/ekam\"\n },\n \"contributes\": {\n \"configuration\": {\n \"type\": \"object\",\n \"title\": \"ekam configuration\",\n \"properties\": {\n \"ekam.path\": {\n \"type\": \"string\",\n \"default\": \"ekam-langserve\",\n \"description\": \"The path to ekam-langserve executable, e.g.: /usr/bin/ekam-langserve\"\n },\n \"ekam.arguments\": {\n \"type\": \"array\",\n \"default\": [\n \"localhost:41315\"\n ],\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"Arguments for ekam-lsp server; should specify host:port of Ekam\"\n }\n }\n }\n }\n}\n" }, { "path": "vscode/src/extension.ts", "content": "import * as vscode from 'vscode';\nimport * as vscodelc from 'vscode-languageclient';\n\n/**\n * Method to get workspace configuration option\n * @param option name of the option (e.g. for ekam.path should be path)\n * @param defaultValue default value to return if option is not set\n */\nfunction getConfig(option: string, defaultValue?: any): T {\n const config = vscode.workspace.getConfiguration('ekam');\n return config.get(option, defaultValue);\n}\n\n/**\n * this method is called when your extension is activate\n * your extension is activated the very first time the command is executed\n */\nexport function activate(context: vscode.ExtensionContext) {\n const syncFileEvents = getConfig('syncFileEvents', true);\n\n const options: vscodelc.Executable = {\n command: getConfig('path'),\n args: getConfig('arguments')\n };\n const serverOptions: vscodelc.ServerOptions = options;\n\n const clientOptions: vscodelc.LanguageClientOptions = {\n documentSelector: [{ scheme: 'file' }]\n };\n\n const ekamClient = new vscodelc.LanguageClient('Ekam Language Server', serverOptions, clientOptions);\n console.log('Ekam Language Server is now active!');\n\n const disposable = ekamClient.start();\n context.subscriptions.push(disposable);\n}\n" }, { "path": "vscode/tsconfig.json", "content": "{\n \"compilerOptions\": {\n \"module\": \"commonjs\",\n \"target\": \"es6\",\n \"outDir\": \"out\",\n \"lib\": [\n \"es6\",\n \"es2015.core\",\n \"es2015.collection\",\n \"es2015.generator\",\n \"es2015.iterable\",\n \"es2015.promise\",\n \"es2015.symbol\",\n \"es2016.array.include\"\n ],\n \"sourceMap\": true,\n \"rootDir\": \".\",\n \"alwaysStrict\": true,\n \"noEmitOnError\": true,\n \"noFallthroughCasesInSwitch\": true,\n \"noImplicitAny\": true,\n \"noImplicitReturns\": true,\n \"noImplicitThis\": true\n },\n \"exclude\": [\n \"node_modules\",\n \".vscode-test\"\n ]\n}" } ]