[ { "path": ".github/ISSUE_TEMPLATE/bug_report.md", "content": "---\nname: Bug report\nabout: Create a report to help us improve\nlabels: \n\n---\n\n**Describe the bug**\nA clear and concise description of what the bug is.\n\n**To Reproduce**\n\nSteps to reproduce the behavior:\n1. Go to '...'\n2. Click on '....'\n3. Scroll down to '....'\n4. See error\n\n**Please always include the output of the following commands in your report!**\n\n* ``brig bug -s``\n\n**Expected behavior**\nA clear and concise description of what you expected to happen.\n\n**Additional context**\nAdd any other context about the problem here.\n" }, { "path": ".github/ISSUE_TEMPLATE/feature_request.md", "content": "---\nname: Feature request\nabout: Suggest an idea for this project\nlabels: \n\n---\n\n**Is your feature request related to a problem? Please describe.**\nA clear and concise description of what the problem is. Ex. I'm always frustrated when [...]\n\n**Describe the solution you'd like**\nA clear and concise description of what you want to happen.\n\n**Describe alternatives you've considered**\nA clear and concise description of any alternative solutions or features you've considered.\n\n**Additional context**\nAdd any other context or screenshots about the feature request here.\n\n\n**Please keep in mind that new features should be orthogonal (i.e. complement) to the existing features.**\n" }, { "path": ".gitignore", "content": "brig\n*.coverprofile\ncoverage.out\n_vendor*\nTODO\n.idea\nmage\nipfs\nrepo/setup/ipfs\n.task\ntags\ncov.out\n" }, { "path": ".mailmap", "content": "Christopher Pahl \nChristopher Pahl \nChristopher Pahl \nChristopher Pahl \nChristopher Pahl \nChristopher Pahl \nChristopher Pahl \n" }, { "path": ".travis.yml", "content": "language: go\nsudo: required\ngo:\n - \"1.15\"\nnotifications:\n email:\n - sahib@online.de\ninstall:\n - sudo apt-get install fuse capnproto\n - mkdir -p ${GOPATH}/bin\n - export GOBIN=${GOPATH}/bin\n - export PATH=\"${GOPATH}/bin:${PATH}\"\n - export GO111MODULE=on\n - go get -u github.com/rakyll/gotest\n - go get -u github.com/phogolabs/parcello\n - go get -u github.com/phogolabs/parcello/cmd/parcello\n - go get -u zombiezen.com/go/capnproto2/...\n - go get -u github.com/go-task/task/v3/cmd/task\n - wget https://dist.ipfs.io/go-ipfs/v0.7.0/go-ipfs_v0.7.0_linux-amd64.tar.gz -O /tmp/ipfs.tgz\n - tar -C /tmp -xvf /tmp/ipfs.tgz\n - cp /tmp/go-ipfs/ipfs $GOBIN\n - export PATH=\"${GOPATH}/bin:${PATH}\"\n - task\n\nscript:\n - export PATH=\"${GOPATH}/bin:${PATH}\"\n - travis_wait 30 bash scripts/run-tests.sh\n" }, { "path": "CHANGELOG.md", "content": "# Change Log\n\nAll notable changes to this project will be documented in this file.\n\nThe format follows [keepachangelog.com]. Please stick to it.\n\n## [0.5.3] -- 2020-07-20\n\nDrastic speed up of listing and show operation.\n\nIn previous version simple IsCached operation on 500 MB file was taking\nmore than 30 seconds. The reason is splitting a file in chunks of no more\nthan 256 kB in ipfs. It took time to establish connection to ipfs and check\nstatus of every chunk. The new caching scheme of intermediate results helps\nto avoid unnecessary connection.\n\nThere is an additional heuristic: if a reference/hash stores less or equal\nto 262158 bytes, then this hash will not have children links (hashes). This\nseems to be true for IPFS up to v0.6.0 but there is no guarantee that it\nwill be true in further version.\n\nThe initial check is now very fast, less than second for the same 500 MB\nfile. Also, recursive check does not rerun full check on probed files now,\nso this is also done much faster. The down side: the caching result are\nstale and can be misreported for up to 5 minutes (with current cache\nexpiration settings).\n\n### Changed\n\nAdded caching mechanism for ipfs interaction.\n\n## [0.5.2] -- 2020-07-16\n\nBug fix release.\n\n### Fixed\n\n- Report correct cache status for a hash with multiple children links.\n The IsCached reported as yes for a large files (>256 kB with ipfs backend)\n since such files are split into multiple blocks. Strangely, the parent\n node is somehow precached without asking (maybe it happens when brig checks\n for the backend size), but its children are not unless we pin or read file \n content.\n\n### Changed\n\n- `brig ls /file.name` will return listing for a single file. Before it worked \n only with directories. Now it behaves similar to the standard file system \n listing (ls) command.\n\n## [0.5.1] -- 2020-07-15\n\nImprovements and bug fixes in the fuse layer. The fuse layer consequent read is\nfactor of 20 faster now.\n\n### Fixed\n\n- Fix reading larger than 64 kB files. The read was from limitedStream with\n 64 kB size. It was spitting EOF when end of the buffer was hit, which is\n not the same as true end of file.\n\n### Changed\n\n- Fuse file handle keeps track of the seek position. So, in consequent\n reads it does not have to reseek the stream which costs a lot of time.\n On my machine the speed went up from about 200 kB/s to 5 MB/s. It is\n still much slower than direct read from disk (30 MB/s) but probably\n expected due to ipfs, compression, and encryption layers.\n\n## [0.5.0] -- 2020-07-13\n\nThis version is mostly bug fixes of unreleased version 0.4.2 by Chris Pahl,\nwho is the original author and maintainer of the »brig«. Output of the diff and \nsync command is now different from the behaviour outlined in the old manual.\nThere are also fixes to make fuse mounts to work properly. Compatibility wise,\nmetadata exchange expects to see the cachedSize field for proper handling,\nolder versions do not provide it.\nSo, I think it justifies the bump in the minor version.\n\nTODO: documentation does not reflect all the changes.\n\n### Fixed\n\n- Fix the behaviour of fast forward changes (i.e. when the file changed only at \n one location).\n- Fix the merge over deleted (ghost) file\n- Fix handling of the resurrected at remote entries.\n- Fix gateway 'get' for authorized user\n- Fix gateway to block download/view files outside of permitted folders\n- Fix bug with pinning at sync. The program used source node info to pin \n instead of destination. So a random node at destination was pinned.\n- Fix bug in repinner size calculator, which could end up with negative numbers\n in unsigned integer and thus report a crazy high values.\n- Fix in capnp template to be compatible with more modern version of capnp\n- Fix handling the pin state border case, when our data report a pin but ipfs \n is not. We ask to repin at ipfs backend.\n- Fix the destination pin status preservation during merge.\n- Multiple fixes in the file system frontend (fuse).\n - Correct file sizes which propagate to the directory size\n - Make sure attributes of a file are not stale\n - Redone writing and staging handling. No if we open file for writing\n we store the modified content in memory. When we flush to the backend\n we compare content change with this memory stored content. Old way did not\n catch such changes.\n - Touch can do create new file over the ghost nodes.\n - We can move/rename things within fuse mount via brig backend.\n TODO: there is small time (several seconds), when file info is not picked \n by fuse after rename. It might be a bug in fuse library itself. It is not\n too critical right now.\n\n### Changed\n\n- Repinner can unpin explicitly pinned files, if they scheduled for deletion \n (i.e. beyond min-max version requirement). Otherwise, we will have stale old\n versions which will take storage space forever.\n- Diff and Sync show the direction of the merge. Diff takes precedence \n according to the modification time. TODO: I need to add new conflict\n resolution strategy: chronological/time and use it only if required.\n I felt chronological strategy is more natural way for syncing different \n repos, so I program time resolution as default.\n- Changes are specific to files not directories. If you create a directory\n in one repo and do the same on the other, it is not a conflict unless, there\n are files with different content. Otherwise, it can be easily merged.\n- Preserve source modification time when merging.\n- Better display of missing or removed entries.\n- When syncing create patches for every commit, before the patch was done \n between old known commit and the current state. That was breaking possibly\n non conflicting merges, since the common parent was lost during the commit \n skip. Sync is a bit longer now, but I think it worse it.\n- Modules are compatible with go v1.14\n- Shorten time format for »brig ls«\n\n### Added\n\n- New option: fs.repin.pin_unpinned. When set to 'false' it saves traffic and \n does not pin what is already pinned. When 'true' pins files within pinning \n requirements (the old behavior); this essentially pre-caches files at the \n backend but uses traffic and disk space.\n- New option: fs.sync.pin_added. If false the sync does not pin/cache at sync\n new/added at remote files. This is handy if we want only sync the metadata \n but not the content itself. I.e. bandwidth saving mode. Opposite (the old \n behavior) is also possible if we want to sync the content and are not concerned\n about bandwidth.\n- Help for »brig gateway add«.\n- Added cached size information to listings, show, and internal file or \n directory info. Technically, it is the same as size of backend, since the \n cached size could be zero at a given time (TODO rename accordingly).\n- The cached size is transmitted via capnp.\n\n## [0.4.1 Capricious Clownfish] -- 2019-03-31\n\nA smaller release with some bug fixes and a few new features. Also one bigger\nstability and speed improvement. Thanks to everyone that gave feedback!\n\n### Fixed\n\n- Fix two badger db related crashes that lead to a crash in the daemon. One was\n related to having nested transactions, the one was related to having an open\n iterator while committing data to the database.\n- Fix some dependencies that led to errors for some users (thanks @vasket)\n- The gateway code now tries to reconnect the websocket whenever it was closed\n due to bad connectivity or similar issues. This led to a state where files\n were only updated after reloading the page.\n- Several smaller fixes in the remotes view, i.e. the owner name was displayed\n wrong and most of the settings could not be set outside the test environment.\n Also the diff output was different in the UI and brig diff.\n- We now error out early if e.g. »brig ls« was issued, but there is no repo.\n Before it tried to start a daemon and waited a long time before timing out.\n- Made »brig mkdir« always prefix a »/« to a path which would lead to funny\n issues otherwise.\n\n### Added\n\n- Add a --offline flag to the following subcommands: ``cat``, ``tar``,\n ``mount`` and ``fstab add``. These flags will only output files that are\n locally cached and will not cause timeouts therefore. Trying other files will\n result in an error.\n- »brig show« now outputs if a file/directory is locally cached. This is not\n the same as pinned, since you can pin a file but it might not be cached yet.\n- Make the gateway host all of its JavaScript, fonts and CSS code itself by\n baking it into the binary. This will enable people running the gateway in\n environments where no internet connection is available to reach the CDN used\n before.\n- Add the possibility to copy the fingerprint in the UI via a button click.\n Before the fingerprint was shown over two lines which made copying tricky.\n- A PKGBUILD for ArchLinux was added, which builds ``brig`` from the\n ``develop`` branch. Thanks @vasket!\n\n### Changed\n\n- The ``brig remote ls`` command no longer does active I/O between nodes to check\n if a node is authenticated. Instead it relies on info from the peer server\n which can apply better caching. The peer server is also able to use information\n from dials and requests to/from other peers to update the ping information.\n- Switch the internal checksum algorithm to ``blake2s-256`` from ``sha3-256``.\n This change was made for speed reasons and leads to a slightly different looking\n checksum format in the command line output. This change MIGHT lead to incompatibilities.\n- Also swap ``scrypt`` with ``argon2`` for key derivation and lower the hashing settings\n until acceptable performance was achieved.\n- Replace the Makefile with a magefile, i.e. a build script written in Go only which has\n no dependencies and can bootstrap itself.\n- Include IPFS config output in »brig bug«.\n\n### Removed\n\n* The old Makefile was removed and replaced with a Go only solution.\n\n## [0.4.0 Capricious Clownfish] -- 2019-03-19\n\nIt's only been a few months since the last release (December 2018), but there\nare a ton of new features / general changes that total in about 15k added lines\nof code. The biggest changes are definitely refactoring IPFS into its own\nprocess and providing a nice UI written in Elm. But those are just two of the\nbiggest ones, see the full list below.\n\nAs always, ``brig`` is **always looking for contributors.** Anything from\nfeedback to pull requests is greatly appreciated.\n\n### Fixed\n\n- Many documentation fixes and updates.\n- Gateway: Prefer server cipher suites over client's choice.\n- Gateway: Make sure to enable timeouts.\n- Bugfix in catfs that could lead to truncated file streams.\n* Lower the memory hunger of BadgerDB.\n* Fix a bug that stopped badger transactions when they got too big.\n\n### Added\n\n* The IPFS daemon does not live in the ``brig`` process itself anymore.\n It can now use any existing / running IPFS daemon. If ``ipfs`` is not installed,\n it will download a local copy and setup a repository in the default place.\n Notice that this is a completely backwards-incompatible change.\n\n* New UI: The Gateway feature was greatly extended and an UI was developed that\n exposes many features in an easily usable way to people that are used to a\n Dropbox like interface. See\n [here](https://brig.readthedocs.io/en/develop/tutorial/gateway.html) for some\n screenshots of the UI and documentation on how to set it up. The gateway\n supports users with different roles (``admin``, ``editor``, ``collaborator``,\n ``viewer``, ``link-only``) and also supports logging as anonymous user (not by\n default!). You can also limit what users can see which folders.\n\n* New event subsystems. This enables users to receive updates in \"realtime\"\n from other remotes. This is built on top of the experimental pubsub feature\n of IPFS and thus needs a daemon that was started with\n ``--enable-pubsub-experiment``. Users can decide to receive updates from\n a remote by issuing ``brig remote auto-update enable ``. [More\n details in the documentation](https://brig.readthedocs.io/en/develop/tutorial/remotes.html#automatic-updating).\n\n* Change the way pinning works. ``brig`` will not unpin old versions anymore,\n but leave that to the [repinning settings](https://brig.readthedocs.io/en/develop/tutorial/pinning.html#repinning).\n This is an automatic process that will make sure to keep at least ``x``\n versions, unpin all versions greater than ``y`` and make sure that only a\n certain filesystem quota is used.\n\n* New ``trash`` subcommand that makes it easy to show deleted files (``brig\n trash ls``) and undelete them again (``brig trash undelete ``).\n\n* New ``brig push`` command to ask a remote to sync with us. For this to work\n the remote needs to allow this to us via ``brig remote auto-push enable ``. See also the\n [documentation](https://brig.readthedocs.io/en/develop/tutorial/remotes.html#pushing-changes).\n\n* New way to handle conflicts: ``embrace`` will always pick the version of the remote you are syncing with.\n This is especially useful if you are building an archival node where you can push changes to.\n See also the [documentation](https://brig.readthedocs.io/en/develop/tutorial/remotes.html#conflicts).\n You can configure the conflict strategy now either globally, per remote or for a specific folder.\n\n* Read only folders. Those are folders that can be shared with others, but when\n we synchronize with them, the folder is exempted from any modifications.\n\n* Implement automated invocation of the garbage collector of IPFS. By default\n it is called once per hour and will clean up files that were unpinned. Note\n that this will also unpin files that are not owned by ``brig``! If you don't want this,\n you should use a separate IPFS instance for ``brig``.\n\n* It's now possible to create ``.tar`` files that are filtered by certain patterns.\n This functionality is currently only exposed in the gateway, not in the command line.\n\n* Easier debugging by having a ``pprof`` server open by default (until we\n consider the daemon to be stable enough to disable it by default). You can get\n a performance graph of the last 30s by issuing ``go tool pprof -web\n \"http://localhost:$(brig d p)/debug/pprof/profile?seconds=30\"``\n\n* One way install script to easily get a ``brig`` binary in seconds on your computer:\n ``bash <(curl -s https://raw.githubusercontent.com/sahib/brig/master/scripts/install.sh)``\n\n### Changed\n\n* Starting with this release we will provide pre-compiled binaries for the most common platforms on the [release page](https://github.com/sahib/brig/releases).\n* Introduce proper linting process (``make lint``)\n* ``init`` will now set some IPFS config values that improve connectivity and performance\n of ``brig``. You can disable this via ``--no-ipfs-optimization``.\n* Disable pre-caching by default due to extreme slow-ness.\n* Migrate to ``go mod`` since we do not need to deal with ``gx`` packages anymore.\n* There is no (broken) ``make install`` target anymore. Simply do ``make`` and\n ``sudo cp brig /usr/local/bin`` or wherever you want to put it.\n\n### Removed\n\n* A lot of old code that was there to support running IPFS inside the daemon process.\n As a side effect, ``brig`` is now much snappier.\n\n## [0.3.0 Galloping Galapagos] -- 2018-12-07\n\n### Fixed\n\n- Compression guessing is now using Go's http.DetectContentType()\n\n### Added\n\n* New gateway subcommand and feature. Now files and directories can be easily\n shared to non-brig users via a normal webserver. Also includes easy https setup.\n\n### Changed\n\n### Removed\n\n### Deprecated\n\n## [0.2.0 Baffling Buck] -- 2018-11-21\n\n### Fixed\n\nAll features mentioned in the documentation should work now.\n\n### Added\n\nMany new features, including password management, partial diffs and partial syncing.\n\n### Changed\n\nMany internal things. Too many to list in this early stage.\n\n### Removed\n\nNothing substantial.\n\n### Deprecated\n\nNothing.\n\n## [0.1.0 Analphabetic Antelope] -- 2018-04-21\n\nInitial release on the Linux Info Day 2018 in Augsburg.\n\n[unreleased]: https://github.com/sahib/rmlint/compare/master...develop\n[0.1.0]: https://github.com/sahib/brig/releases/tag/v0.1.0\n[keepachangelog.com]: http://keepachangelog.com/\n" }, { "path": "Dockerfile", "content": "FROM golang\nMAINTAINER sahib@online.de\n\n# Most test cases can use the pre-defined BRIG_PATH.\nENV BRIG_PATH /var/repo\nRUN mkdir -p $BRIG_PATH\nENV BRIG_USER=\"charlie@wald.de/container\"\n\n# Build the brig binary:\nENV BRIG_SOURCE /go/src/github.com/sahib/brig\nENV BRIG_BINARY_PATH /usr/bin/brig\nCOPY . $BRIG_SOURCE\nWORKDIR $BRIG_SOURCE\nRUN make\n\n# Download IPFS, so the container can startup faster.\n# (brig can also download the binary for you, but later)\nRUN wget https://dist.ipfs.io/go-ipfs/v0.4.19/go-ipfs_v0.4.19_linux-amd64.tar.gz -O /tmp/ipfs.tar.gz\nRUN tar xfv /tmp/ipfs.tar.gz -C /tmp\nRUN cp /tmp/go-ipfs/ipfs /usr/bin\n\nEXPOSE 6666\nEXPOSE 4001\n\nCOPY scripts/docker-normal-startup.sh /bin/run.sh\nCMD [\"/bin/bash\", \"/bin/run.sh\"]\n" }, { "path": "LICENSE", "content": " GNU AFFERO GENERAL PUBLIC LICENSE\n Version 3, 19 November 2007\n\n Copyright (C) 2007 Free Software Foundation, Inc. \n Everyone is permitted to copy and distribute verbatim copies\n of this license document, but changing it is not allowed.\n\n Preamble\n\n The GNU Affero General Public License is a free, copyleft license for\nsoftware and other kinds of works, specifically designed to ensure\ncooperation with the community in the case of network server software.\n\n The licenses for most software and other practical works are designed\nto take away your freedom to share and change the works. By contrast,\nour General Public Licenses are intended to guarantee your freedom to\nshare and change all versions of a program--to make sure it remains free\nsoftware for all its users.\n\n When we speak of free software, we are referring to freedom, not\nprice. Our General Public Licenses are designed to make sure that you\nhave the freedom to distribute copies of free software (and charge for\nthem if you wish), that you receive source code or can get it if you\nwant it, that you can change the software or use pieces of it in new\nfree programs, and that you know you can do these things.\n\n Developers that use our General Public Licenses protect your rights\nwith two steps: (1) assert copyright on the software, and (2) offer\nyou this License which gives you legal permission to copy, distribute\nand/or modify the software.\n\n A secondary benefit of defending all users' freedom is that\nimprovements made in alternate versions of the program, if they\nreceive widespread use, become available for other developers to\nincorporate. Many developers of free software are heartened and\nencouraged by the resulting cooperation. However, in the case of\nsoftware used on network servers, this result may fail to come about.\nThe GNU General Public License permits making a modified version and\nletting the public access it on a server without ever releasing its\nsource code to the public.\n\n The GNU Affero General Public License is designed specifically to\nensure that, in such cases, the modified source code becomes available\nto the community. It requires the operator of a network server to\nprovide the source code of the modified version running there to the\nusers of that server. Therefore, public use of a modified version, on\na publicly accessible server, gives the public access to the source\ncode of the modified version.\n\n An older license, called the Affero General Public License and\npublished by Affero, was designed to accomplish similar goals. This is\na different license, not a version of the Affero GPL, but Affero has\nreleased a new version of the Affero GPL which permits relicensing under\nthis license.\n\n The precise terms and conditions for copying, distribution and\nmodification follow.\n\n TERMS AND CONDITIONS\n\n 0. Definitions.\n\n \"This License\" refers to version 3 of the GNU Affero General Public License.\n\n \"Copyright\" also means copyright-like laws that apply to other kinds of\nworks, such as semiconductor masks.\n\n \"The Program\" refers to any copyrightable work licensed under this\nLicense. Each licensee is addressed as \"you\". \"Licensees\" and\n\"recipients\" may be individuals or organizations.\n\n To \"modify\" a work means to copy from or adapt all or part of the work\nin a fashion requiring copyright permission, other than the making of an\nexact copy. The resulting work is called a \"modified version\" of the\nearlier work or a work \"based on\" the earlier work.\n\n A \"covered work\" means either the unmodified Program or a work based\non the Program.\n\n To \"propagate\" a work means to do anything with it that, without\npermission, would make you directly or secondarily liable for\ninfringement under applicable copyright law, except executing it on a\ncomputer or modifying a private copy. Propagation includes copying,\ndistribution (with or without modification), making available to the\npublic, and in some countries other activities as well.\n\n To \"convey\" a work means any kind of propagation that enables other\nparties to make or receive copies. Mere interaction with a user through\na computer network, with no transfer of a copy, is not conveying.\n\n An interactive user interface displays \"Appropriate Legal Notices\"\nto the extent that it includes a convenient and prominently visible\nfeature that (1) displays an appropriate copyright notice, and (2)\ntells the user that there is no warranty for the work (except to the\nextent that warranties are provided), that licensees may convey the\nwork under this License, and how to view a copy of this License. If\nthe interface presents a list of user commands or options, such as a\nmenu, a prominent item in the list meets this criterion.\n\n 1. Source Code.\n\n The \"source code\" for a work means the preferred form of the work\nfor making modifications to it. \"Object code\" means any non-source\nform of a work.\n\n A \"Standard Interface\" means an interface that either is an official\nstandard defined by a recognized standards body, or, in the case of\ninterfaces specified for a particular programming language, one that\nis widely used among developers working in that language.\n\n The \"System Libraries\" of an executable work include anything, other\nthan the work as a whole, that (a) is included in the normal form of\npackaging a Major Component, but which is not part of that Major\nComponent, and (b) serves only to enable use of the work with that\nMajor Component, or to implement a Standard Interface for which an\nimplementation is available to the public in source code form. A\n\"Major Component\", in this context, means a major essential component\n(kernel, window system, and so on) of the specific operating system\n(if any) on which the executable work runs, or a compiler used to\nproduce the work, or an object code interpreter used to run it.\n\n The \"Corresponding Source\" for a work in object code form means all\nthe source code needed to generate, install, and (for an executable\nwork) run the object code and to modify the work, including scripts to\ncontrol those activities. However, it does not include the work's\nSystem Libraries, or general-purpose tools or generally available free\nprograms which are used unmodified in performing those activities but\nwhich are not part of the work. For example, Corresponding Source\nincludes interface definition files associated with source files for\nthe work, and the source code for shared libraries and dynamically\nlinked subprograms that the work is specifically designed to require,\nsuch as by intimate data communication or control flow between those\nsubprograms and other parts of the work.\n\n The Corresponding Source need not include anything that users\ncan regenerate automatically from other parts of the Corresponding\nSource.\n\n The Corresponding Source for a work in source code form is that\nsame work.\n\n 2. Basic Permissions.\n\n All rights granted under this License are granted for the term of\ncopyright on the Program, and are irrevocable provided the stated\nconditions are met. This License explicitly affirms your unlimited\npermission to run the unmodified Program. The output from running a\ncovered work is covered by this License only if the output, given its\ncontent, constitutes a covered work. This License acknowledges your\nrights of fair use or other equivalent, as provided by copyright law.\n\n You may make, run and propagate covered works that you do not\nconvey, without conditions so long as your license otherwise remains\nin force. You may convey covered works to others for the sole purpose\nof having them make modifications exclusively for you, or provide you\nwith facilities for running those works, provided that you comply with\nthe terms of this License in conveying all material for which you do\nnot control copyright. Those thus making or running the covered works\nfor you must do so exclusively on your behalf, under your direction\nand control, on terms that prohibit them from making any copies of\nyour copyrighted material outside their relationship with you.\n\n Conveying under any other circumstances is permitted solely under\nthe conditions stated below. Sublicensing is not allowed; section 10\nmakes it unnecessary.\n\n 3. Protecting Users' Legal Rights From Anti-Circumvention Law.\n\n No covered work shall be deemed part of an effective technological\nmeasure under any applicable law fulfilling obligations under article\n11 of the WIPO copyright treaty adopted on 20 December 1996, or\nsimilar laws prohibiting or restricting circumvention of such\nmeasures.\n\n When you convey a covered work, you waive any legal power to forbid\ncircumvention of technological measures to the extent such circumvention\nis effected by exercising rights under this License with respect to\nthe covered work, and you disclaim any intention to limit operation or\nmodification of the work as a means of enforcing, against the work's\nusers, your or third parties' legal rights to forbid circumvention of\ntechnological measures.\n\n 4. Conveying Verbatim Copies.\n\n You may convey verbatim copies of the Program's source code as you\nreceive it, in any medium, provided that you conspicuously and\nappropriately publish on each copy an appropriate copyright notice;\nkeep intact all notices stating that this License and any\nnon-permissive terms added in accord with section 7 apply to the code;\nkeep intact all notices of the absence of any warranty; and give all\nrecipients a copy of this License along with the Program.\n\n You may charge any price or no price for each copy that you convey,\nand you may offer support or warranty protection for a fee.\n\n 5. Conveying Modified Source Versions.\n\n You may convey a work based on the Program, or the modifications to\nproduce it from the Program, in the form of source code under the\nterms of section 4, provided that you also meet all of these conditions:\n\n a) The work must carry prominent notices stating that you modified\n it, and giving a relevant date.\n\n b) The work must carry prominent notices stating that it is\n released under this License and any conditions added under section\n 7. This requirement modifies the requirement in section 4 to\n \"keep intact all notices\".\n\n c) You must license the entire work, as a whole, under this\n License to anyone who comes into possession of a copy. This\n License will therefore apply, along with any applicable section 7\n additional terms, to the whole of the work, and all its parts,\n regardless of how they are packaged. This License gives no\n permission to license the work in any other way, but it does not\n invalidate such permission if you have separately received it.\n\n d) If the work has interactive user interfaces, each must display\n Appropriate Legal Notices; however, if the Program has interactive\n interfaces that do not display Appropriate Legal Notices, your\n work need not make them do so.\n\n A compilation of a covered work with other separate and independent\nworks, which are not by their nature extensions of the covered work,\nand which are not combined with it such as to form a larger program,\nin or on a volume of a storage or distribution medium, is called an\n\"aggregate\" if the compilation and its resulting copyright are not\nused to limit the access or legal rights of the compilation's users\nbeyond what the individual works permit. Inclusion of a covered work\nin an aggregate does not cause this License to apply to the other\nparts of the aggregate.\n\n 6. Conveying Non-Source Forms.\n\n You may convey a covered work in object code form under the terms\nof sections 4 and 5, provided that you also convey the\nmachine-readable Corresponding Source under the terms of this License,\nin one of these ways:\n\n a) Convey the object code in, or embodied in, a physical product\n (including a physical distribution medium), accompanied by the\n Corresponding Source fixed on a durable physical medium\n customarily used for software interchange.\n\n b) Convey the object code in, or embodied in, a physical product\n (including a physical distribution medium), accompanied by a\n written offer, valid for at least three years and valid for as\n long as you offer spare parts or customer support for that product\n model, to give anyone who possesses the object code either (1) a\n copy of the Corresponding Source for all the software in the\n product that is covered by this License, on a durable physical\n medium customarily used for software interchange, for a price no\n more than your reasonable cost of physically performing this\n conveying of source, or (2) access to copy the\n Corresponding Source from a network server at no charge.\n\n c) Convey individual copies of the object code with a copy of the\n written offer to provide the Corresponding Source. This\n alternative is allowed only occasionally and noncommercially, and\n only if you received the object code with such an offer, in accord\n with subsection 6b.\n\n d) Convey the object code by offering access from a designated\n place (gratis or for a charge), and offer equivalent access to the\n Corresponding Source in the same way through the same place at no\n further charge. You need not require recipients to copy the\n Corresponding Source along with the object code. If the place to\n copy the object code is a network server, the Corresponding Source\n may be on a different server (operated by you or a third party)\n that supports equivalent copying facilities, provided you maintain\n clear directions next to the object code saying where to find the\n Corresponding Source. Regardless of what server hosts the\n Corresponding Source, you remain obligated to ensure that it is\n available for as long as needed to satisfy these requirements.\n\n e) Convey the object code using peer-to-peer transmission, provided\n you inform other peers where the object code and Corresponding\n Source of the work are being offered to the general public at no\n charge under subsection 6d.\n\n A separable portion of the object code, whose source code is excluded\nfrom the Corresponding Source as a System Library, need not be\nincluded in conveying the object code work.\n\n A \"User Product\" is either (1) a \"consumer product\", which means any\ntangible personal property which is normally used for personal, family,\nor household purposes, or (2) anything designed or sold for incorporation\ninto a dwelling. In determining whether a product is a consumer product,\ndoubtful cases shall be resolved in favor of coverage. For a particular\nproduct received by a particular user, \"normally used\" refers to a\ntypical or common use of that class of product, regardless of the status\nof the particular user or of the way in which the particular user\nactually uses, or expects or is expected to use, the product. A product\nis a consumer product regardless of whether the product has substantial\ncommercial, industrial or non-consumer uses, unless such uses represent\nthe only significant mode of use of the product.\n\n \"Installation Information\" for a User Product means any methods,\nprocedures, authorization keys, or other information required to install\nand execute modified versions of a covered work in that User Product from\na modified version of its Corresponding Source. The information must\nsuffice to ensure that the continued functioning of the modified object\ncode is in no case prevented or interfered with solely because\nmodification has been made.\n\n If you convey an object code work under this section in, or with, or\nspecifically for use in, a User Product, and the conveying occurs as\npart of a transaction in which the right of possession and use of the\nUser Product is transferred to the recipient in perpetuity or for a\nfixed term (regardless of how the transaction is characterized), the\nCorresponding Source conveyed under this section must be accompanied\nby the Installation Information. But this requirement does not apply\nif neither you nor any third party retains the ability to install\nmodified object code on the User Product (for example, the work has\nbeen installed in ROM).\n\n The requirement to provide Installation Information does not include a\nrequirement to continue to provide support service, warranty, or updates\nfor a work that has been modified or installed by the recipient, or for\nthe User Product in which it has been modified or installed. Access to a\nnetwork may be denied when the modification itself materially and\nadversely affects the operation of the network or violates the rules and\nprotocols for communication across the network.\n\n Corresponding Source conveyed, and Installation Information provided,\nin accord with this section must be in a format that is publicly\ndocumented (and with an implementation available to the public in\nsource code form), and must require no special password or key for\nunpacking, reading or copying.\n\n 7. Additional Terms.\n\n \"Additional permissions\" are terms that supplement the terms of this\nLicense by making exceptions from one or more of its conditions.\nAdditional permissions that are applicable to the entire Program shall\nbe treated as though they were included in this License, to the extent\nthat they are valid under applicable law. If additional permissions\napply only to part of the Program, that part may be used separately\nunder those permissions, but the entire Program remains governed by\nthis License without regard to the additional permissions.\n\n When you convey a copy of a covered work, you may at your option\nremove any additional permissions from that copy, or from any part of\nit. (Additional permissions may be written to require their own\nremoval in certain cases when you modify the work.) You may place\nadditional permissions on material, added by you to a covered work,\nfor which you have or can give appropriate copyright permission.\n\n Notwithstanding any other provision of this License, for material you\nadd to a covered work, you may (if authorized by the copyright holders of\nthat material) supplement the terms of this License with terms:\n\n a) Disclaiming warranty or limiting liability differently from the\n terms of sections 15 and 16 of this License; or\n\n b) Requiring preservation of specified reasonable legal notices or\n author attributions in that material or in the Appropriate Legal\n Notices displayed by works containing it; or\n\n c) Prohibiting misrepresentation of the origin of that material, or\n requiring that modified versions of such material be marked in\n reasonable ways as different from the original version; or\n\n d) Limiting the use for publicity purposes of names of licensors or\n authors of the material; or\n\n e) Declining to grant rights under trademark law for use of some\n trade names, trademarks, or service marks; or\n\n f) Requiring indemnification of licensors and authors of that\n material by anyone who conveys the material (or modified versions of\n it) with contractual assumptions of liability to the recipient, for\n any liability that these contractual assumptions directly impose on\n those licensors and authors.\n\n All other non-permissive additional terms are considered \"further\nrestrictions\" within the meaning of section 10. If the Program as you\nreceived it, or any part of it, contains a notice stating that it is\ngoverned by this License along with a term that is a further\nrestriction, you may remove that term. If a license document contains\na further restriction but permits relicensing or conveying under this\nLicense, you may add to a covered work material governed by the terms\nof that license document, provided that the further restriction does\nnot survive such relicensing or conveying.\n\n If you add terms to a covered work in accord with this section, you\nmust place, in the relevant source files, a statement of the\nadditional terms that apply to those files, or a notice indicating\nwhere to find the applicable terms.\n\n Additional terms, permissive or non-permissive, may be stated in the\nform of a separately written license, or stated as exceptions;\nthe above requirements apply either way.\n\n 8. Termination.\n\n You may not propagate or modify a covered work except as expressly\nprovided under this License. Any attempt otherwise to propagate or\nmodify it is void, and will automatically terminate your rights under\nthis License (including any patent licenses granted under the third\nparagraph of section 11).\n\n However, if you cease all violation of this License, then your\nlicense from a particular copyright holder is reinstated (a)\nprovisionally, unless and until the copyright holder explicitly and\nfinally terminates your license, and (b) permanently, if the copyright\nholder fails to notify you of the violation by some reasonable means\nprior to 60 days after the cessation.\n\n Moreover, your license from a particular copyright holder is\nreinstated permanently if the copyright holder notifies you of the\nviolation by some reasonable means, this is the first time you have\nreceived notice of violation of this License (for any work) from that\ncopyright holder, and you cure the violation prior to 30 days after\nyour receipt of the notice.\n\n Termination of your rights under this section does not terminate the\nlicenses of parties who have received copies or rights from you under\nthis License. If your rights have been terminated and not permanently\nreinstated, you do not qualify to receive new licenses for the same\nmaterial under section 10.\n\n 9. Acceptance Not Required for Having Copies.\n\n You are not required to accept this License in order to receive or\nrun a copy of the Program. Ancillary propagation of a covered work\noccurring solely as a consequence of using peer-to-peer transmission\nto receive a copy likewise does not require acceptance. However,\nnothing other than this License grants you permission to propagate or\nmodify any covered work. These actions infringe copyright if you do\nnot accept this License. Therefore, by modifying or propagating a\ncovered work, you indicate your acceptance of this License to do so.\n\n 10. Automatic Licensing of Downstream Recipients.\n\n Each time you convey a covered work, the recipient automatically\nreceives a license from the original licensors, to run, modify and\npropagate that work, subject to this License. You are not responsible\nfor enforcing compliance by third parties with this License.\n\n An \"entity transaction\" is a transaction transferring control of an\norganization, or substantially all assets of one, or subdividing an\norganization, or merging organizations. If propagation of a covered\nwork results from an entity transaction, each party to that\ntransaction who receives a copy of the work also receives whatever\nlicenses to the work the party's predecessor in interest had or could\ngive under the previous paragraph, plus a right to possession of the\nCorresponding Source of the work from the predecessor in interest, if\nthe predecessor has it or can get it with reasonable efforts.\n\n You may not impose any further restrictions on the exercise of the\nrights granted or affirmed under this License. For example, you may\nnot impose a license fee, royalty, or other charge for exercise of\nrights granted under this License, and you may not initiate litigation\n(including a cross-claim or counterclaim in a lawsuit) alleging that\nany patent claim is infringed by making, using, selling, offering for\nsale, or importing the Program or any portion of it.\n\n 11. Patents.\n\n A \"contributor\" is a copyright holder who authorizes use under this\nLicense of the Program or a work on which the Program is based. The\nwork thus licensed is called the contributor's \"contributor version\".\n\n A contributor's \"essential patent claims\" are all patent claims\nowned or controlled by the contributor, whether already acquired or\nhereafter acquired, that would be infringed by some manner, permitted\nby this License, of making, using, or selling its contributor version,\nbut do not include claims that would be infringed only as a\nconsequence of further modification of the contributor version. For\npurposes of this definition, \"control\" includes the right to grant\npatent sublicenses in a manner consistent with the requirements of\nthis License.\n\n Each contributor grants you a non-exclusive, worldwide, royalty-free\npatent license under the contributor's essential patent claims, to\nmake, use, sell, offer for sale, import and otherwise run, modify and\npropagate the contents of its contributor version.\n\n In the following three paragraphs, a \"patent license\" is any express\nagreement or commitment, however denominated, not to enforce a patent\n(such as an express permission to practice a patent or covenant not to\nsue for patent infringement). To \"grant\" such a patent license to a\nparty means to make such an agreement or commitment not to enforce a\npatent against the party.\n\n If you convey a covered work, knowingly relying on a patent license,\nand the Corresponding Source of the work is not available for anyone\nto copy, free of charge and under the terms of this License, through a\npublicly available network server or other readily accessible means,\nthen you must either (1) cause the Corresponding Source to be so\navailable, or (2) arrange to deprive yourself of the benefit of the\npatent license for this particular work, or (3) arrange, in a manner\nconsistent with the requirements of this License, to extend the patent\nlicense to downstream recipients. \"Knowingly relying\" means you have\nactual knowledge that, but for the patent license, your conveying the\ncovered work in a country, or your recipient's use of the covered work\nin a country, would infringe one or more identifiable patents in that\ncountry that you have reason to believe are valid.\n\n If, pursuant to or in connection with a single transaction or\narrangement, you convey, or propagate by procuring conveyance of, a\ncovered work, and grant a patent license to some of the parties\nreceiving the covered work authorizing them to use, propagate, modify\nor convey a specific copy of the covered work, then the patent license\nyou grant is automatically extended to all recipients of the covered\nwork and works based on it.\n\n A patent license is \"discriminatory\" if it does not include within\nthe scope of its coverage, prohibits the exercise of, or is\nconditioned on the non-exercise of one or more of the rights that are\nspecifically granted under this License. You may not convey a covered\nwork if you are a party to an arrangement with a third party that is\nin the business of distributing software, under which you make payment\nto the third party based on the extent of your activity of conveying\nthe work, and under which the third party grants, to any of the\nparties who would receive the covered work from you, a discriminatory\npatent license (a) in connection with copies of the covered work\nconveyed by you (or copies made from those copies), or (b) primarily\nfor and in connection with specific products or compilations that\ncontain the covered work, unless you entered into that arrangement,\nor that patent license was granted, prior to 28 March 2007.\n\n Nothing in this License shall be construed as excluding or limiting\nany implied license or other defenses to infringement that may\notherwise be available to you under applicable patent law.\n\n 12. No Surrender of Others' Freedom.\n\n If conditions are imposed on you (whether by court order, agreement or\notherwise) that contradict the conditions of this License, they do not\nexcuse you from the conditions of this License. If you cannot convey a\ncovered work so as to satisfy simultaneously your obligations under this\nLicense and any other pertinent obligations, then as a consequence you may\nnot convey it at all. For example, if you agree to terms that obligate you\nto collect a royalty for further conveying from those to whom you convey\nthe Program, the only way you could satisfy both those terms and this\nLicense would be to refrain entirely from conveying the Program.\n\n 13. Remote Network Interaction; Use with the GNU General Public License.\n\n Notwithstanding any other provision of this License, if you modify the\nProgram, your modified version must prominently offer all users\ninteracting with it remotely through a computer network (if your version\nsupports such interaction) an opportunity to receive the Corresponding\nSource of your version by providing access to the Corresponding Source\nfrom a network server at no charge, through some standard or customary\nmeans of facilitating copying of software. This Corresponding Source\nshall include the Corresponding Source for any work covered by version 3\nof the GNU General Public License that is incorporated pursuant to the\nfollowing paragraph.\n\n Notwithstanding any other provision of this License, you have\npermission to link or combine any covered work with a work licensed\nunder version 3 of the GNU General Public License into a single\ncombined work, and to convey the resulting work. The terms of this\nLicense will continue to apply to the part which is the covered work,\nbut the work with which it is combined will remain governed by version\n3 of the GNU General Public License.\n\n 14. Revised Versions of this License.\n\n The Free Software Foundation may publish revised and/or new versions of\nthe GNU Affero General Public License from time to time. Such new versions\nwill be similar in spirit to the present version, but may differ in detail to\naddress new problems or concerns.\n\n Each version is given a distinguishing version number. If the\nProgram specifies that a certain numbered version of the GNU Affero General\nPublic License \"or any later version\" applies to it, you have the\noption of following the terms and conditions either of that numbered\nversion or of any later version published by the Free Software\nFoundation. If the Program does not specify a version number of the\nGNU Affero General Public License, you may choose any version ever published\nby the Free Software Foundation.\n\n If the Program specifies that a proxy can decide which future\nversions of the GNU Affero General Public License can be used, that proxy's\npublic statement of acceptance of a version permanently authorizes you\nto choose that version for the Program.\n\n Later license versions may give you additional or different\npermissions. However, no additional obligations are imposed on any\nauthor or copyright holder as a result of your choosing to follow a\nlater version.\n\n 15. Disclaimer of Warranty.\n\n THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY\nAPPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT\nHOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM \"AS IS\" WITHOUT WARRANTY\nOF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,\nTHE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR\nPURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM\nIS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF\nALL NECESSARY SERVICING, REPAIR OR CORRECTION.\n\n 16. Limitation of Liability.\n\n IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING\nWILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS\nTHE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY\nGENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE\nUSE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF\nDATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD\nPARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),\nEVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF\nSUCH DAMAGES.\n\n 17. Interpretation of Sections 15 and 16.\n\n If the disclaimer of warranty and limitation of liability provided\nabove cannot be given local legal effect according to their terms,\nreviewing courts shall apply local law that most closely approximates\nan absolute waiver of all civil liability in connection with the\nProgram, unless a warranty or assumption of liability accompanies a\ncopy of the Program in return for a fee.\n\n END OF TERMS AND CONDITIONS\n\n How to Apply These Terms to Your New Programs\n\n If you develop a new program, and you want it to be of the greatest\npossible use to the public, the best way to achieve this is to make it\nfree software which everyone can redistribute and change under these terms.\n\n To do so, attach the following notices to the program. It is safest\nto attach them to the start of each source file to most effectively\nstate the exclusion of warranty; and each file should have at least\nthe \"copyright\" line and a pointer to where the full notice is found.\n\n \n Copyright (C) \n\n This program is free software: you can redistribute it and/or modify\n it under the terms of the GNU Affero General Public License as published by\n the Free Software Foundation, either version 3 of the License, or\n (at your option) any later version.\n\n This program is distributed in the hope that it will be useful,\n but WITHOUT ANY WARRANTY; without even the implied warranty of\n MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n GNU Affero General Public License for more details.\n\n You should have received a copy of the GNU Affero General Public License\n along with this program. If not, see .\n\nAlso add information on how to contact you by electronic and paper mail.\n\n If your software can interact with users remotely through a computer\nnetwork, you should also make sure that it provides a way for users to\nget its source. For example, if your program is a web application, its\ninterface could display a \"Source\" link that leads users to an archive\nof the code. There are many ways you could offer source, and different\nsolutions will be better for different programs; see section 13 for the\nspecific requirements.\n\n You should also get your employer (if you work as a programmer) or school,\nif any, to sign a \"copyright disclaimer\" for the program, if necessary.\nFor more information on this, and how to apply and follow the GNU AGPL, see\n.\n" }, { "path": "PULL_REQUEST_TEMPLATE.md", "content": "Here's a small checklist before publishing your pull request:\n\n* Did you ``go fmt`` all code?\n* Does your code style fit with the rest of the code base?\n* Did you run ``go run mage.go dev:lint``?\n* Did you write tests if necessary?\n* Did you consider if changes to the docs are necessary?\n* Did you check if you need something to CHANGELOG.md?\n\n\nThank you for your contribution.\n" }, { "path": "README.md", "content": "# `brig`: Ship your data around the world\n\n
\n\"a\n
\n\n[![go reportcard](https://goreportcard.com/badge/github.com/sahib/brig)](https://goreportcard.com/report/github.com/sahib/brig)\n[![GoDoc](https://godoc.org/github.com/sahib/brig?status.svg)](https://godoc.org/github.com/sahib/brig)\n[![Build Status](https://travis-ci.org/sahib/brig.svg?branch=master)](https://travis-ci.org/sahib/brig)\n[![Documentation](https://readthedocs.org/projects/rmlint/badge/?version=latest)](http://brig.readthedocs.io/en/latest)\n[![License: AGPL v3](https://img.shields.io/badge/License-AGPL%20v3-blue.svg)](https://www.gnu.org/licenses/agpl-3.0)\n[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/1558/badge)](https://bestpractices.coreinfrastructure.org/en/projects/1558)\n\n![brig gateway in the files tab](docs/_static/gateway-files.png)\n\n## Table of Contents\n\n- [`brig`: Ship your data around the world](#brig-ship-your-data-around-the-world)\n - [Table of Contents](#table-of-contents)\n - [About](#about)\n - [Installation](#installation)\n - [Getting started](#getting-started)\n - [Status](#status)\n - [Documentation](#documentation)\n - [Donations](#donations)\n - [Focus](#focus)\n\n## About\n\n`brig` is a distributed & secure file synchronization tool with version control.\nIt is based on `IPFS`, written in Go and will feel familiar to `git` users.\n\n**Key feature highlights:**\n\n* Encryption of data in rest and transport + compression on the fly.\n* Simplified `git` version control.\n* Sync algorithm that can handle moved files and empty directories and files.\n* Your data does not need to be stored on the device you are currently using.\n* FUSE filesystem that feels like a normal (sync) folder.\n* No central server at all. Still, central architectures can be build with `brig`.\n* Simple user identification and discovery with users that look like email addresses.\n\nAlso take a look [at the documentation](http://brig.readthedocs.io/en/latest/index.html) for more details.\n\n## Installation\n\nYou can download the latest script with the following oneliner:\n\n```bash\n# Before you execute this, ask yourself if you trust me.\n$ bash <(curl -s https://raw.githubusercontent.com/sahib/brig/master/scripts/install.sh)\n```\n\nAlternatively, you can simply grab the latest binary from the [release tab](https://github.com/sahib/brig/releases).\n\nDevelopment versions can be installed easily by compiling yourself. If you have\na recent version of `go` (`>= 1.10`) installed, it should be as easy as this:\n\n```bash\n$ go get -d -v -u github.com/sahib/brig # Download the sources.\n$ cd $GOPATH/src/github.com/sahib/brig # Go to the source directory.\n$ git checkout develop # Checkout the develop branch.\n$ go run mage.go # Build the software.\n$ $GOPATH/bin/brig help # Run the binary.\n```\n\nPlease refer to the [install docs](https://brig.readthedocs.io/en/latest/installation.html) for more details.\n\n## Getting started\n\n[![asciicast](https://asciinema.org/a/163713.png)](https://asciinema.org/a/163713)\n\n...If you want to know, what to do after you can read the\n[Quickstart](http://brig.readthedocs.io/en/latest/quickstart.html).\n\nThere is also a ``#brig`` room on ``matrix.org`` you can join with any [Matrix](https://matrix.org) client.\nClick [this link](https://riot.im/app/#/room/#brig:matrix.org) to join the room directly via [Riot.im](https://about.riot.im).\n\n## Status\n\nThis software is in a **beta phase** currently. All mentioned features should\nwork. Things might still change rapidly and there will be no guarantees given\nbefore version `1.0.0`. Do not use `brig` yet as only storage for your\nproduction data. There are still bugs, but it should be safe enough to toy\naround with it quite a bit.\n\nThis project has started end of 2015 and has seen many conceptual changes in\nthe meantime. It started out as research project. After writing my [master\ntheses](https://github.com/disorganizer/brig-thesis) on it, it was put down for\na few months until I picked at up again and currently am trying to push it to\nusable software.\n\nIf you want to open a bug report, just type `brig bug` to get a readily filled template for you.\n\n## Documentation\n\nAll documentation can be found on [ReadTheDocs.org](http://brig.readthedocs.io/en/latest/index.html).\n\n## Donations\n\nIf you're interested in the development and would think about supporting me\nfinancially, then please [contact me!](mailto:sahib@online.de) If you'd like to\ngive me a small & steady donation, you can always use *Liberapay*:\n\n\n\n*Thank you!*\n" }, { "path": "Taskfile.yml", "content": "# This file controls how brig is build.\n# It is a nicer to use alternative to Makefiles.\n# Please read the documentation over at:\n#\n# https://taskfile.dev\n#\n# The actual commands that do the work are written in bash.\n# See the scripts/ folder for them.\n#\n# When changing the structure of the repository, please remember\n# to update the \"sources\" list in this file if dependencies\n# of a build target were added, removed or changed.\nversion: '3'\n\ntasks:\n default:\n deps: [build]\n\n elm:\n desc: \"Compile elm sources to Javascript\"\n cmds:\n - cd gateway/elm && elm make src/Main.elm --output ../static/js/app.js\n sources:\n - ./gateway/elm/**/*.elm\n generates:\n - ./gateway/static/js/app.js\n method: checksum\n summary: |\n Build the elm frontend.\n\n generate:\n desc: \"Generate build dependencies\"\n cmds:\n - scripts/generate.sh\n sources:\n - scripts/generate.sh\n - ./**/*.capnp\n - ./gateway/static/**/**/**/**\n\n build:\n deps: [generate]\n desc: \"Build the brig binary\"\n cmds:\n - ./scripts/build.sh\n sources:\n - ./scripts/build.sh\n - go.mod\n - ./*.go\n - ./**/*.go\n\n test:\n desc: \"Run integration & unit tests\"\n cmds:\n - bash scripts/run-tests.sh\n\n lint:\n desc: \"Run static linters on the code\"\n cmds:\n - bash scripts/run-linter.sh\n\n sloc:\n desc: \"Count the lines of code\"\n cmds:\n - bash scripts/count-lines-of-code.sh\n" }, { "path": "autocomplete/bash_autocomplete", "content": "#!/bin/bash\n\n# This should be installed to /etc/bash_completion.d/brig and sourced.\n# If you want to try out the autocompletion, just source this file.\n_cli_bash_autocomplete() {\n local cur opts base\n COMPREPLY=()\n cur=\"${COMP_WORDS[COMP_CWORD]}\"\n opts=$( ${COMP_WORDS[@]:0:$COMP_CWORD} --generate-bash-completion )\n COMPREPLY=( $(compgen -W \"${opts}\" -- ${cur}) )\n return 0\n}\n\ncomplete -F _cli_bash_autocomplete brig\n" }, { "path": "autocomplete/zsh_autocomplete", "content": "_cli_zsh_autocomplete() {\n local -a opts\n opts=(\"${(@f)$(_CLI_ZSH_AUTOCOMPLETE_HACK=1 ${words[@]:0:#words[@]-1} --generate-bash-completion)}\")\n _describe 'values' opts\n return\n}\n\ncompdef _cli_zsh_autocomplete brig\n" }, { "path": "backend/backend.go", "content": "package backend\n\nimport (\n\t\"errors\"\n\t\"io\"\n\t\"os\"\n\n\t\"github.com/sahib/brig/backend/httpipfs\"\n\t\"github.com/sahib/brig/backend/mock\"\n\t\"github.com/sahib/brig/catfs\"\n\teventsBackend \"github.com/sahib/brig/events/backend\"\n\tnetBackend \"github.com/sahib/brig/net/backend\"\n\t\"github.com/sahib/brig/repo\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\nvar (\n\t// ErrNoSuchBackend is returned when passing an invalid backend name\n\tErrNoSuchBackend = errors.New(\"No such backend\")\n)\n\n// VersionInfo is a small interface that will return version info about the\n// backend.\ntype VersionInfo interface {\n\tSemVer() string\n\tName() string\n\tRev() string\n}\n\n// Backend is a amalgamation of all backend interfaces required for brig to work.\ntype Backend interface {\n\trepo.Backend\n\tcatfs.FsBackend\n\tnetBackend.Backend\n\teventsBackend.Backend\n}\n\n// ForwardLogByName will forward the logs of the backend `name` to `w`.\nfunc ForwardLogByName(name string, w io.Writer) error {\n\tswitch name {\n\tcase \"httpipfs\":\n\t\treturn nil\n\tcase \"mock\":\n\t\treturn nil\n\t}\n\n\treturn ErrNoSuchBackend\n}\n\n// FromName returns a suitable backend for a human readable name.\n// If an invalid name is passed, nil is returned.\nfunc FromName(name, path, fingerprint string) (Backend, error) {\n\tswitch name {\n\tcase \"httpipfs\":\n\t\treturn httpipfs.NewNode(path, fingerprint)\n\tcase \"mock\":\n\t\tuser := \"alice\"\n\t\tif envUser := os.Getenv(\"BRIG_MOCK_USER\"); envUser != \"\" {\n\t\t\tuser = envUser\n\t\t}\n\n\t\tif envNetDbPath := os.Getenv(\"BRIG_MOCK_NET_DB_PATH\"); envNetDbPath != \"\" {\n\t\t\tpath = envNetDbPath\n\t\t}\n\n\t\treturn mock.NewMockBackend(path, user), nil\n\t}\n\n\treturn nil, ErrNoSuchBackend\n}\n\n// Version returns version info for the backend `name`.\nfunc Version(name, path string) VersionInfo {\n\tswitch name {\n\tcase \"mock\":\n\t\treturn mock.Version()\n\tcase \"httpipfs\":\n\t\tnd, err := httpipfs.NewNode(path, \"\")\n\t\tif err != nil {\n\t\t\tlog.Debugf(\"failed to get version\")\n\t\t\treturn nil\n\t\t}\n\n\t\tdefer nd.Close()\n\t\treturn nd.Version()\n\tdefault:\n\t\treturn nil\n\t}\n}\n" }, { "path": "backend/httpipfs/gc.go", "content": "package httpipfs\n\nimport (\n\t\"bufio\"\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\n\te \"github.com/pkg/errors\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// GC will trigger the garbage collector of IPFS.\n// Cleaned up hashes will be returned as a list\n// (note that those hashes are not always ours)\nfunc (nd *Node) GC() ([]h.Hash, error) {\n\tctx := context.Background()\n\tresp, err := nd.sh.Request(\"repo/gc\").Send(ctx)\n\n\tif err != nil {\n\t\treturn nil, e.Wrapf(resp.Error, \"gc request\")\n\t}\n\n\tdefer resp.Close()\n\n\tif resp.Error != nil {\n\t\treturn nil, e.Wrapf(resp.Error, \"gc resp\")\n\t}\n\n\ths := []h.Hash{}\n\tbr := bufio.NewReader(resp.Output)\n\tfor {\n\t\tline, err := br.ReadBytes('\\n')\n\t\tif err != nil {\n\t\t\tbreak\n\t\t}\n\n\t\traw := struct {\n\t\t\tKey map[string]string\n\t\t}{}\n\n\t\tlr := bytes.NewReader(line)\n\t\tif err := json.NewDecoder(lr).Decode(&raw); err != nil {\n\t\t\treturn nil, e.Wrapf(err, \"json decode\")\n\t\t}\n\n\t\tfor _, cid := range raw.Key {\n\t\t\th, err := h.FromB58String(cid)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, e.Wrapf(err, \"gc: hash decode\")\n\t\t\t}\n\n\t\t\ths = append(hs, h)\n\t\t}\n\t}\n\n\tlog.Debugf(\"GC returned %d hashes\", len(hs))\n\treturn hs, nil\n}\n" }, { "path": "backend/httpipfs/gc_test.go", "content": "package httpipfs\n\nimport (\n\t\"bytes\"\n\t\"testing\"\n\n\t\"github.com/sahib/brig/util/testutil\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestGC(t *testing.T) {\n\tt.Skipf(\"will be replaced by bash based e2e tests\")\n\n\tWithIpfs(t, 1, func(t *testing.T, ipfsPath string) {\n\t\tnd, err := NewNode(ipfsPath, \"\")\n\t\trequire.Nil(t, err)\n\n\t\tdata := testutil.CreateDummyBuf(4096 * 1024)\n\t\thash, err := nd.Add(bytes.NewReader(data))\n\t\trequire.Nil(t, err)\n\n\t\trequire.Nil(t, nd.Unpin(hash))\n\t\thashes, err := nd.GC()\n\t\trequire.Nil(t, err)\n\t\trequire.True(t, len(hashes) > 0)\n\t})\n}\n" }, { "path": "backend/httpipfs/io.go", "content": "package httpipfs\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"sync\"\n\n\tshell \"github.com/ipfs/go-ipfs-api\"\n\t\"github.com/sahib/brig/catfs/mio\"\n\th \"github.com/sahib/brig/util/hashlib\"\n)\n\nfunc cat(s *shell.Shell, path string, offset int64) (io.ReadCloser, error) {\n\trb := s.Request(\"cat\", path)\n\trb.Option(\"offset\", offset)\n\tresp, err := rb.Send(context.Background())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif resp.Error != nil {\n\t\treturn nil, resp.Error\n\t}\n\n\treturn resp.Output, nil\n}\n\ntype streamWrapper struct {\n\tmu sync.Mutex\n\n\tio.ReadCloser\n\tnd *Node\n\thash h.Hash\n\toff int64\n\tsize int64\n}\n\nfunc (sw *streamWrapper) Read(buf []byte) (int, error) {\n\tsw.mu.Lock()\n\tdefer sw.mu.Unlock()\n\n\tn, err := sw.ReadCloser.Read(buf)\n\tif err != nil {\n\t\treturn n, err\n\t}\n\n\tsw.off += int64(n)\n\treturn n, err\n}\n\nfunc (sw *streamWrapper) WriteTo(w io.Writer) (int64, error) {\n\tsw.mu.Lock()\n\tdefer sw.mu.Unlock()\n\n\treturn io.Copy(w, sw)\n}\n\nfunc (sw *streamWrapper) cachedSize() (int64, error) {\n\tctx := context.Background()\n\tresp, err := sw.nd.sh.Request(\n\t\t\"files/stat\",\n\t\t\"/ipfs/\"+sw.hash.B58String(),\n\t).Send(ctx)\n\n\tif err != nil {\n\t\treturn -1, err\n\t}\n\n\tdefer resp.Close()\n\n\tif resp.Error != nil {\n\t\treturn -1, resp.Error\n\t}\n\n\traw := struct {\n\t\tSize int64\n\t}{}\n\n\tif err := json.NewDecoder(resp.Output).Decode(&raw); err != nil {\n\t\treturn -1, err\n\t}\n\n\treturn raw.Size, nil\n}\n\nfunc (sw *streamWrapper) getAbsOffset(offset int64, whence int) (int64, error) {\n\tswitch whence {\n\tcase io.SeekStart:\n\t\tsw.off = offset\n\t\treturn offset, nil\n\tcase io.SeekCurrent:\n\t\tsw.off += offset\n\t\treturn sw.off, nil\n\tcase io.SeekEnd:\n\t\tsize, err := sw.cachedSize()\n\t\tif err != nil {\n\t\t\treturn -1, err\n\t\t}\n\n\t\tsw.off = size + offset\n\t\treturn sw.off, nil\n\tdefault:\n\t\treturn -1, fmt.Errorf(\"invalid whence: %v\", whence)\n\t}\n}\n\n// TODO: Seek is currently freaking expensive.\n// Does IPFS maybe offer a better way to do this?\nfunc (sw *streamWrapper) Seek(offset int64, whence int) (int64, error) {\n\tsw.mu.Lock()\n\tdefer sw.mu.Unlock()\n\n\tabsOffset, err := sw.getAbsOffset(offset, whence)\n\tif err != nil {\n\t\treturn -1, err\n\t}\n\n\trc, err := cat(sw.nd.sh, sw.hash.B58String(), absOffset)\n\tif err != nil {\n\t\treturn -1, err\n\t}\n\n\tif sw.ReadCloser != nil {\n\t\t// Not sure if that is even needed...\n\t\t// TODO: measure memory consumption and see if we can do\n\t\t// without discarding left over bytes.\n\t\tgo func(rc io.ReadCloser) {\n\t\t\tio.Copy(ioutil.Discard, rc)\n\t\t\trc.Close()\n\t\t}(sw.ReadCloser)\n\t}\n\n\tsw.off = absOffset\n\tsw.ReadCloser = rc\n\treturn absOffset, nil\n}\n\n// Cat returns a stream associated with `hash`.\nfunc (nd *Node) Cat(hash h.Hash) (mio.Stream, error) {\n\trc, err := cat(nd.sh, hash.B58String(), 0)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &streamWrapper{\n\t\tnd: nd,\n\t\thash: hash,\n\t\tReadCloser: rc,\n\t\toff: 0,\n\t\tsize: -1,\n\t}, nil\n}\n\n// Add puts the contents of `r` into IPFS and returns its hash.\nfunc (nd *Node) Add(r io.Reader) (h.Hash, error) {\n\ths, err := nd.sh.Add(r)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn h.FromB58String(hs)\n}\n" }, { "path": "backend/httpipfs/io_test.go", "content": "package httpipfs\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"testing\"\n\n\t\"github.com/sahib/brig/util/testutil\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestAddCatBasic(t *testing.T) {\n\tt.Skipf(\"will be replaced by bash based e2e tests\")\n\n\tWithIpfs(t, 1, func(t *testing.T, ipfsPath string) {\n\t\tnd, err := NewNode(ipfsPath, \"\")\n\t\trequire.Nil(t, err)\n\n\t\tdata := testutil.CreateDummyBuf(4096 * 1024)\n\t\thash, err := nd.Add(bytes.NewReader(data))\n\t\trequire.Nil(t, err)\n\n\t\tfmt.Println(hash)\n\n\t\tstream, err := nd.Cat(hash)\n\t\trequire.Nil(t, err)\n\n\t\techoData, err := ioutil.ReadAll(stream)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, data, echoData)\n\t})\n}\n\nfunc TestAddCatSize(t *testing.T) {\n\tt.Skipf(\"will be replaced by bash based e2e tests\")\n\n\tWithIpfs(t, 1, func(t *testing.T, ipfsPath string) {\n\t\tnd, err := NewNode(ipfsPath, \"\")\n\t\trequire.Nil(t, err)\n\n\t\tdata := testutil.CreateDummyBuf(4096 * 1024)\n\t\thash, err := nd.Add(bytes.NewReader(data))\n\t\trequire.Nil(t, err)\n\n\t\tstream, err := nd.Cat(hash)\n\t\trequire.Nil(t, err)\n\n\t\tsize, err := stream.Seek(0, io.SeekEnd)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, int64(len(data)), size)\n\n\t\toff, err := stream.Seek(0, io.SeekStart)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, int64(0), off)\n\n\t\techoData, err := ioutil.ReadAll(stream)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, data, echoData)\n\t})\n}\n" }, { "path": "backend/httpipfs/net.go", "content": "package httpipfs\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"net\"\n\t\"os\"\n\t\"path\"\n\t\"path/filepath\"\n\t\"sync\"\n\t\"time\"\n\n\tshell \"github.com/ipfs/go-ipfs-api\"\n\tnetBackend \"github.com/sahib/brig/net/backend\"\n\t\"github.com/sahib/brig/util\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\ntype connWrapper struct {\n\tnet.Conn\n\n\tpeer string\n\tprotocol string\n\ttargetAddr string\n\tsh *shell.Shell\n}\n\nfunc (cw *connWrapper) LocalAddr() net.Addr {\n\treturn &addrWrapper{\n\t\tprotocol: cw.protocol,\n\t\tpeer: \"\",\n\t}\n}\n\nfunc (cw *connWrapper) RemoteAddr() net.Addr {\n\treturn &addrWrapper{\n\t\tprotocol: cw.protocol,\n\t\tpeer: cw.peer,\n\t}\n}\n\nfunc (cw *connWrapper) Close() error {\n\tdefer cw.Conn.Close()\n\treturn closeStream(cw.sh, cw.protocol, \"\", cw.targetAddr)\n}\n\n// Dial will open a connection to the peer identified by `peerHash`,\n// running `protocol` over it.\nfunc (nd *Node) Dial(peerHash, fingerprint, protocol string) (net.Conn, error) {\n\tif !nd.isOnline() {\n\t\treturn nil, ErrOffline\n\t}\n\n\tself, err := nd.Identity()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif self.Addr == peerHash {\n\t\t// Special case:\n\t\t// When we use the same IPFS daemon for different\n\t\t// brig repositiories, we want still to be able to dial\n\t\t// other brig instances. Since we cannot dial over ipfs\n\t\t// we simply have the port written to /tmp where\n\t\t// we can pick it up on Dial()\n\t\taddr, err := readLocalAddr(peerHash, fingerprint)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn net.Dial(\"tcp\", addr)\n\t}\n\n\tprotocol = path.Join(protocol, peerHash)\n\n\tport := util.FindFreePort()\n\taddr := fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%d\", port)\n\tif err := forward(nd.sh, protocol, addr, peerHash); err != nil {\n\t\treturn nil, err\n\t}\n\n\ttcpAddr := fmt.Sprintf(\"127.0.0.1:%d\", port)\n\tlog.Debugf(\"dial to »%s« over port %d\", peerHash, port)\n\tconn, err := net.Dial(\"tcp\", tcpAddr)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &connWrapper{\n\t\tConn: conn,\n\t\tpeer: peerHash,\n\t\tprotocol: protocol,\n\t\ttargetAddr: addr,\n\t\tsh: nd.sh,\n\t}, nil\n}\n\n//////////////////////////\n\nfunc forward(sh *shell.Shell, protocol, targetAddr, peerID string) error {\n\tctx := context.Background()\n\tpeerID = \"/ipfs/\" + peerID\n\n\trb := sh.Request(\"p2p/forward\", protocol, targetAddr, peerID)\n\trb.Option(\"allow-custom-protocol\", true)\n\tresp, err := rb.Send(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdefer resp.Close()\n\tif resp.Error != nil {\n\t\treturn resp.Error\n\t}\n\n\treturn nil\n}\n\nfunc openListener(sh *shell.Shell, protocol, targetAddr string) error {\n\tctx := context.Background()\n\trb := sh.Request(\"p2p/listen\", protocol, targetAddr)\n\trb.Option(\"allow-custom-protocol\", true)\n\tresp, err := rb.Send(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdefer resp.Close()\n\tif err := resp.Error; err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\nfunc closeStream(sh *shell.Shell, protocol, targetAddr, listenAddr string) error {\n\tctx := context.Background()\n\trb := sh.Request(\"p2p/close\")\n\trb.Option(\"protocol\", protocol)\n\n\tif targetAddr != \"\" {\n\t\trb.Option(\"target-address\", targetAddr)\n\t}\n\n\tif listenAddr != \"\" {\n\t\trb.Option(\"listen-address\", listenAddr)\n\t}\n\n\tresp, err := rb.Send(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdefer resp.Close()\n\tif resp.Error != nil {\n\t\treturn resp.Error\n\t}\n\n\treturn nil\n}\n\ntype addrWrapper struct {\n\tprotocol string\n\tpeer string\n}\n\nfunc (sa *addrWrapper) Network() string {\n\treturn sa.protocol\n}\n\nfunc (sa *addrWrapper) String() string {\n\treturn sa.peer\n}\n\ntype listenerWrapper struct {\n\tlst net.Listener\n\tprotocol string\n\tpeer string\n\ttargetAddr string\n\tfingerprint string\n\tsh *shell.Shell\n}\n\nfunc (lw *listenerWrapper) Accept() (net.Conn, error) {\n\tconn, err := lw.lst.Accept()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &connWrapper{\n\t\tConn: conn,\n\t\tpeer: lw.peer,\n\t\tprotocol: lw.protocol,\n\t\ttargetAddr: lw.targetAddr,\n\t\tsh: lw.sh,\n\t}, nil\n}\n\nfunc (lw *listenerWrapper) Addr() net.Addr {\n\treturn &addrWrapper{\n\t\tprotocol: lw.protocol,\n\t\tpeer: lw.peer,\n\t}\n}\n\nfunc (lw *listenerWrapper) Close() error {\n\tdefer lw.lst.Close()\n\tdefer deleteLocalAddr(lw.peer, lw.fingerprint)\n\treturn closeStream(lw.sh, lw.protocol, lw.targetAddr, \"\")\n}\n\nfunc buildLocalAddrPath(id, fingerprint string) string {\n\treturn filepath.Join(os.TempDir(), fmt.Sprintf(\"brig-%s:%s.addr\", id, fingerprint))\n}\n\nfunc readLocalAddr(id, fingerprint string) (string, error) {\n\tpath := buildLocalAddrPath(id, fingerprint)\n\tdata, err := ioutil.ReadFile(path)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\treturn string(data), nil\n}\n\nfunc deleteLocalAddr(id, fingerprint string) error {\n\tpath := buildLocalAddrPath(id, fingerprint)\n\treturn os.RemoveAll(path)\n}\n\nfunc writeLocalAddr(id, fingerprint, addr string) error {\n\tpath := buildLocalAddrPath(id, fingerprint)\n\treturn ioutil.WriteFile(path, []byte(addr), 0644)\n}\n\n// Listen will listen to the protocol\nfunc (nd *Node) Listen(protocol string) (net.Listener, error) {\n\tif !nd.isOnline() {\n\t\treturn nil, ErrOffline\n\t}\n\n\tself, err := nd.Identity()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// TODO: Is this even needed still?\n\t// Do we want support for having more than one brig per ipfs.\n\t// Append the id to the protocol:\n\tprotocol = path.Join(protocol, self.Addr)\n\n\tport := util.FindFreePort()\n\taddr := fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%d\", port)\n\n\t// Prevent errors by closing any previously opened listeners:\n\tif err := closeStream(nd.sh, protocol, \"\", \"\"); err != nil {\n\t\treturn nil, err\n\t}\n\n\tlog.Debugf(\"backend: listening for %s over port %d\", protocol, port)\n\tif err := openListener(nd.sh, protocol, addr); err != nil {\n\t\treturn nil, err\n\t}\n\n\tlocalAddr := fmt.Sprintf(\"127.0.0.1:%d\", port)\n\tlst, err := net.Listen(\"tcp\", localAddr)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := writeLocalAddr(self.Addr, nd.fingerprint, localAddr); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &listenerWrapper{\n\t\tlst: lst,\n\t\tprotocol: protocol,\n\t\tpeer: self.Addr,\n\t\ttargetAddr: addr,\n\t\tfingerprint: nd.fingerprint,\n\t\tsh: nd.sh,\n\t}, nil\n}\n\n/////////////////////////////////\n\ntype pinger struct {\n\tlastSeen time.Time\n\troundtrip time.Duration\n\terr error\n\n\tmu sync.Mutex\n\tcancel func()\n\tnd *Node\n}\n\n// LastSeen returns the time we pinged the remote last time.\nfunc (p *pinger) LastSeen() time.Time {\n\tp.mu.Lock()\n\tdefer p.mu.Unlock()\n\n\treturn p.lastSeen\n}\n\n// Roundtrip returns the time needed send a single package to\n// the remote and receive the answer.\nfunc (p *pinger) Roundtrip() time.Duration {\n\tp.mu.Lock()\n\tdefer p.mu.Unlock()\n\n\treturn p.roundtrip\n}\n\n// Err will return a non-nil error when the current ping did not succeed.\nfunc (p *pinger) Err() error {\n\tp.mu.Lock()\n\tdefer p.mu.Unlock()\n\n\treturn p.err\n}\n\n// Close will clean up the pinger.\nfunc (p *pinger) Close() error {\n\tif p.cancel != nil {\n\t\tp.cancel()\n\t\tp.cancel = nil\n\t}\n\n\treturn nil\n}\n\nfunc (p *pinger) update(ctx context.Context, addr, self string) {\n\t// Edge case: test setups where we ping ourselves.\n\tif self == addr {\n\t\tp.mu.Lock()\n\t\tp.err = nil\n\t\tp.lastSeen = time.Now()\n\t\tp.roundtrip = time.Duration(0)\n\t\tp.mu.Unlock()\n\t\treturn\n\t}\n\n\t// Do the network op without a lock:\n\troundtrip, err := ping(p.nd.sh, addr)\n\n\tp.mu.Lock()\n\tif err != nil {\n\t\tp.err = err\n\t} else {\n\t\tp.err = nil\n\t\tp.lastSeen = time.Now()\n\t\tp.roundtrip = roundtrip\n\t}\n\n\tp.mu.Unlock()\n}\n\nfunc (p *pinger) Run(ctx context.Context, addr string) error {\n\tself, err := p.nd.Identity()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tp.update(ctx, addr, self.Addr)\n\ttckr := time.NewTicker(10 * time.Second)\n\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\treturn ctx.Err()\n\t\tcase <-tckr.C:\n\t\t\tp.update(ctx, addr, self.Addr)\n\t\t}\n\t}\n}\n\nfunc ping(sh *shell.Shell, peerID string) (time.Duration, error) {\n\tctx := context.Background()\n\tresp, err := sh.Request(\"ping\", peerID).Send(ctx)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\tdefer resp.Close()\n\n\tif resp.Error != nil {\n\t\treturn 0, resp.Error\n\t}\n\n\traw := struct {\n\t\tSuccess bool\n\t\tTime int64\n\t}{}\n\n\tif err := json.NewDecoder(resp.Output).Decode(&raw); err != nil {\n\t\treturn 0, err\n\t}\n\n\tif raw.Success {\n\t\treturn time.Duration(raw.Time), nil\n\t}\n\n\treturn 0, fmt.Errorf(\"no ping\")\n}\n\n// ErrWaiting is the initial error state of a pinger.\n// The error will be unset once a successful ping was made.\nvar ErrWaiting = errors.New(\"waiting for route\")\n\n// Ping will return a pinger for `addr`.\nfunc (nd *Node) Ping(addr string) (netBackend.Pinger, error) {\n\tif !nd.isOnline() {\n\t\treturn nil, ErrOffline\n\t}\n\n\tlog.Debugf(\"backend: start ping »%s«\", addr)\n\tp := &pinger{\n\t\tnd: nd,\n\t\terr: ErrWaiting,\n\t}\n\n\tctx, cancel := context.WithCancel(context.Background())\n\tp.cancel = cancel\n\tgo p.Run(ctx, addr)\n\treturn p, nil\n}\n" }, { "path": "backend/httpipfs/net_test.go", "content": "package httpipfs\n\nimport (\n\t\"bytes\"\n\t\"io\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nconst (\n\tTestProtocol = \"/brig/test/1.0\"\n)\n\nvar (\n\tTestMessage = []byte(\"Hello World!\")\n)\n\nfunc testClientSide(t *testing.T, ipfsPathB string, addr string) {\n\tnd, err := NewNode(ipfsPathB, \"\")\n\trequire.Nil(t, err)\n\n\tconn, err := nd.Dial(addr, \"\", TestProtocol)\n\trequire.Nil(t, err)\n\n\tdefer func() {\n\t\trequire.Nil(t, conn.Close())\n\t}()\n\n\t_, err = conn.Write(TestMessage)\n\trequire.Nil(t, err)\n}\n\nfunc TestDialAndListen(t *testing.T) {\n\tt.Skipf(\"will be replaced by bash based e2e tests\")\n\n\tWithDoubleIpfs(t, 1, func(t *testing.T, ipfsPathA, ipfsPathB string) {\n\t\tnd, err := NewNode(ipfsPathA, \"\")\n\t\trequire.Nil(t, err)\n\n\t\tlst, err := nd.Listen(TestProtocol)\n\t\trequire.Nil(t, err)\n\t\tdefer func() {\n\t\t\trequire.Nil(t, lst.Close())\n\t\t}()\n\n\t\tid, err := nd.Identity()\n\t\trequire.Nil(t, err)\n\n\t\tgo testClientSide(t, ipfsPathB, id.Addr)\n\n\t\tconn, err := lst.Accept()\n\t\trequire.Nil(t, err)\n\n\t\tbuf := &bytes.Buffer{}\n\t\t_, err = io.Copy(buf, conn)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, TestMessage, buf.Bytes())\n\t})\n}\n\nfunc TestPing(t *testing.T) {\n\tt.Skipf(\"will be replaced by bash based e2e tests\")\n\n\tWithDoubleIpfs(t, 1, func(t *testing.T, ipfsPathA, ipfsPathB string) {\n\t\tndA, err := NewNode(ipfsPathA, \"\")\n\t\trequire.NoError(t, err)\n\n\t\tidA, err := ndA.Identity()\n\t\trequire.NoError(t, err)\n\n\t\tpinger, err := ndA.Ping(idA.Addr)\n\t\trequire.NoError(t, err)\n\n\t\tdefer func() {\n\t\t\trequire.NoError(t, pinger.Close())\n\t\t}()\n\n\t\tfor idx := 0; idx < 60; idx++ {\n\t\t\tif pinger.Err() != ErrWaiting {\n\t\t\t\tbreak\n\t\t\t}\n\n\t\t\ttime.Sleep(1 * time.Second)\n\t\t}\n\n\t\trequire.Nil(t, pinger.Err())\n\t\trequire.True(t, pinger.Roundtrip() < time.Second)\n\t\trequire.True(t, time.Since(pinger.LastSeen()) < 2*time.Second)\n\t})\n}\n\nfunc TestDialAndListenOnSingleNode(t *testing.T) {\n\tt.Skipf(\"will be replaced by bash based e2e tests\")\n\n\tWithIpfs(t, 1, func(t *testing.T, ipfsPath string) {\n\t\tnd, err := NewNode(ipfsPath, \"\")\n\t\trequire.Nil(t, err)\n\n\t\tlst, err := nd.Listen(TestProtocol)\n\t\trequire.Nil(t, err)\n\t\tdefer func() {\n\t\t\trequire.Nil(t, lst.Close())\n\t\t}()\n\n\t\tid, err := nd.Identity()\n\t\trequire.Nil(t, err)\n\n\t\tgo testClientSide(t, ipfsPath, id.Addr)\n\n\t\tconn, err := lst.Accept()\n\t\trequire.Nil(t, err)\n\n\t\tbuf := &bytes.Buffer{}\n\t\t_, err = io.Copy(buf, conn)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, TestMessage, buf.Bytes())\n\t})\n}\n\nfunc TestPingSelf(t *testing.T) {\n\tt.Skipf(\"will be replaced by bash based e2e tests\")\n\n\tWithIpfs(t, 1, func(t *testing.T, ipfsPath string) {\n\t\tnd, err := NewNode(ipfsPath, \"\")\n\t\trequire.Nil(t, err)\n\n\t\tid, err := nd.Identity()\n\t\trequire.Nil(t, err)\n\n\t\tpinger, err := nd.Ping(id.Addr)\n\t\trequire.Nil(t, err)\n\n\t\tdefer func() {\n\t\t\trequire.Nil(t, pinger.Close())\n\t\t}()\n\n\t\tfor idx := 0; idx < 60; idx++ {\n\t\t\tif pinger.Err() != ErrWaiting {\n\t\t\t\tbreak\n\t\t\t}\n\n\t\t\ttime.Sleep(250 * time.Millisecond)\n\t\t}\n\n\t\trequire.Nil(t, pinger.Err())\n\t\trequire.True(t, pinger.Roundtrip() < time.Second)\n\t\trequire.True(t, time.Since(pinger.LastSeen()) < 2*time.Second)\n\t})\n}\n" }, { "path": "backend/httpipfs/pin.go", "content": "package httpipfs\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"io\"\n\t\"strings\"\n\n\t\"github.com/patrickmn/go-cache\"\n\th \"github.com/sahib/brig/util/hashlib\"\n)\n\n// IsPinned returns true when `hash` is pinned in some way.\nfunc (nd *Node) IsPinned(hash h.Hash) (bool, error) {\n\tctx := context.Background()\n\tresp, err := nd.sh.Request(\"pin/ls\", hash.B58String()).Send(ctx)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\tdefer resp.Close()\n\n\tif resp.Error != nil {\n\t\tif strings.HasSuffix(resp.Error.Message, \"is not pinned\") {\n\t\t\treturn false, nil\n\t\t}\n\n\t\treturn false, resp.Error\n\t}\n\n\traw := struct {\n\t\tKeys map[string]struct {\n\t\t\tType string\n\t\t}\n\t}{}\n\n\tif err := json.NewDecoder(resp.Output).Decode(&raw); err != nil {\n\t\treturn false, err\n\t}\n\n\tif len(raw.Keys) == 0 {\n\t\treturn false, nil\n\t}\n\n\treturn true, nil\n}\n\n// Pin will pin `hash`.\nfunc (nd *Node) Pin(hash h.Hash) error {\n\treturn nd.sh.Pin(hash.B58String())\n}\n\n// Unpin will unpin `hash`.\nfunc (nd *Node) Unpin(hash h.Hash) error {\n\terr := nd.sh.Unpin(hash.B58String())\n\tif err == nil || err.Error() == \"pin/rm: not pinned or pinned indirectly\" {\n\t\treturn nil\n\t}\n\treturn err\n}\n\ntype objectRef struct {\n\tRef string // hash of the ref\n\tErr string\n}\n\n// Link is a child of a hash.\n// Used by IPFS when files get bigger.\ntype Link struct {\n\tName string\n\tHash string\n\tSize uint64\n}\n\n// IsCached checks if hash and all its children are cached\nfunc (nd *Node) IsCached(hash h.Hash) (bool, error) {\n\tlocallyCached := nd.cache.locallyCached\n\tstat, found := locallyCached.Get(hash.B58String())\n\tif found {\n\t\treturn stat.(bool), nil\n\t}\n\n\t// Nothing in the cache, we have to figure it out.\n\t// We will execute equivalent of\n\t// ipfs refs --offline --recursive hash\n\t// note the `--recursive` switch, we need to check all children links\n\t// if command fails at least one child link/hash is missing\n\tctx := context.Background()\n\treq := nd.sh.Request(\"refs\", hash.B58String())\n\treq.Option(\"offline\", \"true\")\n\treq.Option(\"recursive\", \"true\")\n\tresp, err := req.Send(ctx)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\tdefer resp.Close()\n\tif resp.Error != nil {\n\t\treturn false, resp.Error\n\t}\n\n\tref := objectRef{}\n\tjsonDecoder := json.NewDecoder(resp.Output)\n\tfor {\n\t\tif err := jsonDecoder.Decode(&ref); err == io.EOF {\n\t\t\tbreak\n\t\t} else if err != nil {\n\t\t\treturn false, err\n\t\t}\n\t\tif ref.Err != \"\" {\n\t\t\t// Either main hash or one of its refs/links is not available locally\n\t\t\t// consequently the whole hash is not cached\n\t\t\tlocallyCached.Set(hash.B58String(), false, cache.DefaultExpiration)\n\t\t\treturn false, nil\n\t\t}\n\t}\n\t// if we are here, the parent hash and all its children links/hashes are cached\n\tlocallyCached.Set(hash.B58String(), true, cache.DefaultExpiration)\n\treturn true, nil\n}\n\n// CachedSize returns the cached size of the node.\n// Negative indicates unknow eithe due to error or hash not stored locally\nfunc (nd *Node) CachedSize(hash h.Hash) (int64, error) {\n\tctx := context.Background()\n\treq := nd.sh.Request(\"object/stat\", hash.B58String())\n\t// provides backend size only for cached objects\n\treq.Option(\"offline\", \"true\")\n\tresp, err := req.Send(ctx)\n\tif err != nil {\n\t\treturn -1, err\n\t}\n\n\tdefer resp.Close()\n\n\tif resp.Error != nil {\n\t\treturn -1, resp.Error\n\t}\n\n\traw := struct {\n\t\tCumulativeSize int64\n\t\tKey string\n\t}{}\n\n\tif err := json.NewDecoder(resp.Output).Decode(&raw); err != nil {\n\t\treturn -1, err\n\t}\n\n\treturn raw.CumulativeSize, nil\n}\n" }, { "path": "backend/httpipfs/pin_test.go", "content": "package httpipfs\n\nimport (\n\t\"bytes\"\n\t\"testing\"\n\n\th \"github.com/sahib/brig/util/hashlib\"\n\t\"github.com/sahib/brig/util/testutil\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestPinUnpin(t *testing.T) {\n\tt.Skipf(\"will be replaced by bash based e2e tests\")\n\n\tWithIpfs(t, 1, func(t *testing.T, ipfsPath string) {\n\t\tnd, err := NewNode(ipfsPath, \"\")\n\t\trequire.Nil(t, err)\n\n\t\tdata := testutil.CreateDummyBuf(4096 * 1024)\n\t\thash, err := nd.Add(bytes.NewReader(data))\n\t\trequire.Nil(t, err)\n\n\t\tisPinned, err := nd.IsPinned(hash)\n\t\trequire.Nil(t, err)\n\t\trequire.True(t, isPinned)\n\n\t\trequire.Nil(t, nd.Unpin(hash))\n\n\t\tisPinned, err = nd.IsPinned(hash)\n\t\trequire.Nil(t, err)\n\t\trequire.False(t, isPinned)\n\n\t\trequire.Nil(t, nd.Pin(hash))\n\n\t\tisPinned, err = nd.IsPinned(hash)\n\t\trequire.Nil(t, err)\n\t\trequire.True(t, isPinned)\n\t})\n}\n\nfunc TestIsCached(t *testing.T) {\n\tt.Skipf(\"will be replaced by bash based e2e tests\")\n\n\tWithIpfs(t, 1, func(t *testing.T, ipfsPath string) {\n\t\tnd, err := NewNode(ipfsPath, \"\")\n\t\trequire.Nil(t, err)\n\n\t\thash, err := nd.Add(bytes.NewReader([]byte{1, 2, 3}))\n\t\trequire.Nil(t, err)\n\n\t\tisCached, err := nd.IsCached(hash)\n\t\trequire.Nil(t, err)\n\t\trequire.True(t, isCached)\n\n\t\t// Let's just hope this hash does not exist locally:\n\t\tdummyHash, err := h.FromB58String(\"QmanyEbg6appBzzGaGMZm9NKqPVCbrWaB8ayGDerWh6aMB\")\n\t\trequire.Nil(t, err)\n\n\t\tisCached, err = nd.IsCached(dummyHash)\n\t\trequire.Nil(t, err)\n\t\trequire.False(t, isCached)\n\t})\n}\n" }, { "path": "backend/httpipfs/pubsub.go", "content": "package httpipfs\n\nimport (\n\t\"context\"\n\n\tshell \"github.com/ipfs/go-ipfs-api\"\n\teventsBackend \"github.com/sahib/brig/events/backend\"\n)\n\ntype subWrapper struct {\n\tsub *shell.PubSubSubscription\n}\n\ntype msgWrapper struct {\n\tmsg *shell.Message\n}\n\nfunc (msg *msgWrapper) Data() []byte {\n\treturn msg.msg.Data\n}\n\nfunc (msg *msgWrapper) Source() string {\n\treturn string(msg.msg.From)\n}\n\nfunc (s *subWrapper) Next(ctx context.Context) (eventsBackend.Message, error) {\n\tmsg, err := s.sub.Next()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &msgWrapper{msg: msg}, nil\n}\n\nfunc (s *subWrapper) Close() error {\n\treturn s.sub.Cancel()\n}\n\n// Subscribe will create a subscription for `topic`.\n// You can use the subscription to wait for the next incoming message.\n// This will only work if the daemon supports/has enabled pub sub.\nfunc (nd *Node) Subscribe(ctx context.Context, topic string) (eventsBackend.Subscription, error) {\n\tif !nd.isOnline() {\n\t\treturn nil, ErrOffline\n\t}\n\n\tsub, err := nd.sh.PubSubSubscribe(topic)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &subWrapper{sub: sub}, nil\n}\n\n// PublishEvent will publish `data` on `topic`.\nfunc (nd *Node) PublishEvent(topic string, data []byte) error {\n\tif !nd.isOnline() {\n\t\treturn ErrOffline\n\t}\n\n\treturn nd.sh.PubSubPublish(topic, string(data))\n}\n" }, { "path": "backend/httpipfs/pubsub_test.go", "content": "package httpipfs\n\nimport (\n\t\"context\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestPubSub(t *testing.T) {\n\tt.Skipf(\"will be replaced by bash based e2e tests\")\n\n\t// Only use one ipfs instance, for test performance.\n\tWithIpfs(t, 1, func(t *testing.T, ipfsPath string) {\n\t\tnd, err := NewNode(ipfsPath, \"\")\n\t\trequire.Nil(t, err)\n\n\t\tself, err := nd.Identity()\n\t\trequire.Nil(t, err)\n\n\t\tctx := context.Background()\n\t\tsub, err := nd.Subscribe(ctx, \"test-topic\")\n\t\trequire.Nil(t, err)\n\n\t\tdefer func() {\n\t\t\trequire.Nil(t, sub.Close())\n\t\t}()\n\n\t\ttime.Sleep(1 * time.Second)\n\t\tdata := []byte(\"hello world!\")\n\t\tgo nd.PublishEvent(\"test-topic\", data)\n\n\t\tmsg, err := sub.Next(ctx)\n\t\trequire.Nil(t, err)\n\n\t\trequire.Equal(t, data, msg.Data())\n\t\trequire.Equal(t, self.Addr, msg.Source())\n\t})\n}\n" }, { "path": "backend/httpipfs/resolve.go", "content": "package httpipfs\n\nimport (\n\t\"bufio\"\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\n\tshell \"github.com/ipfs/go-ipfs-api\"\n\tipfsutil \"github.com/ipfs/go-ipfs-util\"\n\tmh \"github.com/multiformats/go-multihash\"\n\t\"github.com/sahib/brig/net/peer\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// PublishName will announce `name` to the network\n// and make us discoverable.\nfunc (nd *Node) PublishName(name string) error {\n\tif !nd.isOnline() {\n\t\treturn ErrOffline\n\t}\n\n\tfullName := \"brig:\" + string(name)\n\tkey, err := nd.sh.BlockPut([]byte(fullName), \"v0\", \"sha2-256\", -1)\n\tlog.Debugf(\"published name: »%s« (key %s)\", name, key)\n\treturn err\n}\n\n// Identity returns our own identity.\n// It will cache the identity after the first request.\nfunc (nd *Node) Identity() (peer.Info, error) {\n\tnd.mu.Lock()\n\tif nd.cachedIdentity != \"\" {\n\t\tdefer nd.mu.Unlock()\n\t\treturn peer.Info{\n\t\t\tName: \"httpipfs\",\n\t\t\tAddr: nd.cachedIdentity,\n\t\t}, nil\n\t}\n\n\t// Do not hold the lock during net ops:\n\tnd.mu.Unlock()\n\n\tid, err := nd.sh.ID()\n\tif err != nil {\n\t\treturn peer.Info{}, err\n\t}\n\n\tnd.mu.Lock()\n\tnd.cachedIdentity = id.ID\n\tnd.mu.Unlock()\n\n\treturn peer.Info{\n\t\tName: \"httpipfs\",\n\t\tAddr: id.ID,\n\t}, nil\n}\n\nfunc findProvider(ctx context.Context, sh *shell.Shell, hash h.Hash) ([]string, error) {\n\tresp, err := sh.Request(\"dht/findprovs\", hash.B58String()).Send(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdefer resp.Output.Close()\n\n\tif resp.Error != nil {\n\t\treturn nil, resp.Error\n\t}\n\n\tids := make(map[string]bool)\n\tbr := bufio.NewReader(resp.Output)\n\tinterrupted := false\n\n\tfor len(ids) < 20 && !interrupted {\n\t\tline, err := br.ReadBytes('\\n')\n\t\tif err != nil {\n\t\t\tbreak\n\t\t}\n\n\t\traw := struct {\n\t\t\tResponses []struct {\n\t\t\t\tID string\n\t\t\t}\n\t\t}{}\n\n\t\tlr := bytes.NewReader(line)\n\t\tif err := json.NewDecoder(lr).Decode(&raw); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tfor _, resp := range raw.Responses {\n\t\t\tids[resp.ID] = true\n\t\t}\n\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\tinterrupted = true\n\t\t\tbreak\n\t\t}\n\t}\n\n\tlinearIDs := []string{}\n\tfor id := range ids {\n\t\tlinearIDs = append(linearIDs, id)\n\t}\n\n\treturn linearIDs, nil\n}\n\n// ResolveName will return all peers that identify themselves as `name`.\n// If ctx is canceled it will return early, but return no error.\nfunc (nd *Node) ResolveName(ctx context.Context, name string) ([]peer.Info, error) {\n\tif !nd.isOnline() {\n\t\treturn nil, ErrOffline\n\t}\n\n\tname = \"brig:\" + name\n\tmhash, err := mh.Sum([]byte(name), ipfsutil.DefaultIpfsHash, -1)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tlog.Debugf(\"backend: resolve »%s« (%s)\", name, mhash.B58String())\n\n\tids, err := findProvider(ctx, nd.sh, h.Hash(mhash))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tinfos := []peer.Info{}\n\tfor _, id := range ids {\n\t\tinfos = append(infos, peer.Info{\n\t\t\tAddr: id,\n\t\t\tName: peer.Name(name),\n\t\t})\n\t}\n\n\treturn infos, nil\n}\n" }, { "path": "backend/httpipfs/resolve_test.go", "content": "package httpipfs\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestPublishResolve(t *testing.T) {\n\tt.Skipf(\"will be replaced by bash based e2e tests\")\n\n\t// Only use one ipfs instance, for test performance.\n\tWithDoubleIpfs(t, 1, func(t *testing.T, ipfsPathA, ipfsPathB string) {\n\t\tndA, err := NewNode(ipfsPathA, \"\")\n\t\trequire.Nil(t, err)\n\n\t\tndB, err := NewNode(ipfsPathB, \"\")\n\t\trequire.Nil(t, err)\n\n\t\t// self, err := ndA.Identity()\n\t\t// require.Nil(t, err)\n\n\t\trequire.Nil(t, ndA.PublishName(\"alice\"))\n\t\tctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)\n\t\tdefer cancel()\n\n\t\tinfos, err := ndB.ResolveName(ctx, \"alice\")\n\t\trequire.Nil(t, err)\n\n\t\t// TODO: This test doesn't produce results yet,\n\t\t// most likely because of time issues (would need to run longer?)\n\t\tfmt.Println(infos)\n\t})\n}\n" }, { "path": "backend/httpipfs/shell.go", "content": "package httpipfs\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/blang/semver\"\n\tshell \"github.com/ipfs/go-ipfs-api\"\n\tma \"github.com/multiformats/go-multiaddr\"\n\t\"github.com/patrickmn/go-cache\"\n\t\"github.com/sahib/brig/repo/setup\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\nvar (\n\t// ErrOffline is returned by operations that need online support\n\t// to work when the backend is in offline mode.\n\tErrOffline = errors.New(\"backend is in offline mode\")\n)\n\n// IpfsStateCache contains various backend related caches\ntype IpfsStateCache struct {\n\tlocallyCached *cache.Cache // shows if the hash and its children is locally cached by ipfs\n}\n\n// Node is the struct that holds the httpipfs backend together.\n// It is a shallow type that has not much own state and is very light.\ntype Node struct {\n\tsh *shell.Shell\n\tmu sync.Mutex\n\tcachedIdentity string\n\tallowNetOps bool\n\tfingerprint string\n\tversion *semver.Version\n\tcache *IpfsStateCache\n\tquiet bool\n}\n\nfunc getExperimentalFeatures(sh *shell.Shell) (map[string]bool, error) {\n\tctx := context.Background()\n\tresp, err := sh.Request(\"config/show\").Send(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdefer resp.Close()\n\n\tif resp.Error != nil {\n\t\treturn nil, resp.Error\n\t}\n\n\traw := struct {\n\t\tExperimental map[string]bool\n\t}{}\n\n\tif err := json.NewDecoder(resp.Output).Decode(&raw); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn raw.Experimental, nil\n}\n\n// Option is a option you can pass to NewNode()\n// It controls the behavior of the node.\ntype Option func(nd *Node)\n\n// WithNoLogging will make the node not print log messages.\n// Useful for commandline use cases.\nfunc WithNoLogging() Option {\n\treturn func(nd *Node) {\n\t\tnd.quiet = true\n\t}\n}\n\nfunc toMultiAddr(ipfsPathOrMultiaddr string) (ma.Multiaddr, error) {\n\tif !filepath.IsAbs(ipfsPathOrMultiaddr) {\n\t\t// multiaddr always start with a slash,\n\t\t// this branch affects only file paths.\n\t\tvar err error\n\t\tipfsPathOrMultiaddr, err = filepath.Abs(ipfsPathOrMultiaddr)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tif _, err := os.Stat(ipfsPathOrMultiaddr); err == nil {\n\t\treturn setup.GetAPIAddrForPath(ipfsPathOrMultiaddr)\n\t}\n\n\treturn ma.NewMultiaddr(ipfsPathOrMultiaddr)\n}\n\n// NewNode returns a new http based IPFS backend.\nfunc NewNode(ipfsPathOrMultiaddr string, fingerprint string, opts ...Option) (*Node, error) {\n\tnd := &Node{\n\t\tallowNetOps: true,\n\t\tfingerprint: fingerprint,\n\t\tcache: &IpfsStateCache{\n\t\t\tlocallyCached: cache.New(5*time.Minute, 10*time.Minute),\n\t\t},\n\t}\n\n\tfor _, opt := range opts {\n\t\topt(nd)\n\t}\n\n\tm, err := toMultiAddr(ipfsPathOrMultiaddr)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif !nd.quiet {\n\t\tlog.Infof(\"Connecting to IPFS HTTP API at %s\", m.String())\n\t}\n\n\tnd.sh = shell.NewShell(m.String())\n\n\tversionString, _, err := nd.sh.Version()\n\tif err != nil && !nd.quiet {\n\t\tlog.Warningf(\"failed to get version: %v\", err)\n\t}\n\n\tversion, err := semver.Parse(versionString)\n\tif err != nil && !nd.quiet {\n\t\tlog.Warningf(\"failed to parse version string of IPFS (»%s«): %v\", versionString, err)\n\t}\n\n\tif !nd.quiet {\n\t\tlog.Infof(\"The IPFS version is »%s«.\", version)\n\t\tif version.LT(semver.MustParse(\"0.4.18\")) {\n\t\t\tlog.Warningf(\"This version is quite old. Please update, if possible.\\n\")\n\t\t\tlog.Warningf(\"We only test on newer versions (>= 0.4.18).\\n\")\n\t\t}\n\t}\n\n\tnd.version = &version\n\n\tif !nd.quiet {\n\t\tfeatures, err := getExperimentalFeatures(nd.sh)\n\t\tif err != nil {\n\t\t\tlog.Warningf(\"Failed to get experimental feature list: %v\", err)\n\t\t} else {\n\t\t\tif !features[\"Libp2pStreamMounting\"] {\n\t\t\t\tlog.Warningf(\"Stream mounting does not seem to be enabled.\")\n\t\t\t\tlog.Warningf(\"Please execute the following to change that:\")\n\t\t\t\tlog.Warningf(\"$ ipfs config --json Experimental.Libp2pStreamMounting true\")\n\t\t\t}\n\t\t}\n\t}\n\n\treturn nd, nil\n}\n\n// IsOnline returns true if the node is in online mode and the daemon is reachable.\nfunc (nd *Node) IsOnline() bool {\n\tnd.mu.Lock()\n\tallowNetOps := nd.allowNetOps\n\tnd.mu.Unlock()\n\n\treturn nd.sh.IsUp() && allowNetOps\n}\n\n// Connect implements Backend.Connect\nfunc (nd *Node) Connect() error {\n\tnd.mu.Lock()\n\tdefer nd.mu.Unlock()\n\n\tnd.allowNetOps = true\n\treturn nil\n}\n\n// Disconnect implements Backend.Disconnect\nfunc (nd *Node) Disconnect() error {\n\tnd.mu.Lock()\n\tdefer nd.mu.Unlock()\n\n\tnd.allowNetOps = false\n\treturn nil\n}\n\nfunc (nd *Node) isOnline() bool {\n\tnd.mu.Lock()\n\tdefer nd.mu.Unlock()\n\n\treturn nd.allowNetOps\n}\n\n// Close implements Backend.Close\nfunc (nd *Node) Close() error {\n\treturn nil\n}\n\n// Name returns \"httpipfs\" as name of the backend.\nfunc (nd *Node) Name() string {\n\treturn \"httpipfs\"\n}\n" }, { "path": "backend/httpipfs/testing.go", "content": "package httpipfs\n\nimport (\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"os/exec\"\n\t\"testing\"\n\t\"time\"\n\n\tshell \"github.com/ipfs/go-ipfs-api\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// WithIpfs starts a new IPFS instance and calls `fn` with the API port to it.\n// `portOff` is the offset to add on all standard ports.\nfunc WithIpfs(t *testing.T, portOff int, fn func(t *testing.T, ipfsPath string)) {\n\tipfsPath, err := ioutil.TempDir(\"\", \"brig-httpipfs-test-\")\n\trequire.Nil(t, err)\n\tdefer os.RemoveAll(ipfsPath)\n\n\tgwtPort := 8081 + portOff\n\tswmPort := 4001 + portOff\n\tapiPort := 5011 + portOff\n\n\tos.Setenv(\"IPFS_PATH\", ipfsPath)\n\tscript := [][]string{\n\t\t{\"ipfs\", \"init\"},\n\t\t{\"ipfs\", \"config\", \"--json\", \"Addresses.Swarm\", fmt.Sprintf(\"[\\\"/ip4/127.0.0.1/tcp/%d\\\"]\", swmPort)},\n\t\t{\"ipfs\", \"config\", \"--json\", \"Experimental.Libp2pStreamMounting\", \"true\"},\n\t\t{\"ipfs\", \"config\", \"Addresses.API\", fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%d\", apiPort)},\n\t\t{\"ipfs\", \"config\", \"Addresses.Gateway\", fmt.Sprintf(\"/ip4/127.0.0.1/tcp/%d\", gwtPort)},\n\t}\n\n\tfor _, line := range script {\n\t\tcmd := exec.Command(line[0], line[1:]...)\n\t\tcmd.Env = append(cmd.Env, fmt.Sprintf(\"IPFS_PATH=%s\", ipfsPath))\n\t\terr := cmd.Run()\n\t\trequire.NoError(t, err)\n\t}\n\n\tdaemonCmd := exec.Command(\"ipfs\", \"daemon\", \"--enable-pubsub-experiment\")\n\t// daemonCmd.Stdout = os.Stdout\n\t// daemonCmd.Stderr = os.Stdout\n\tdaemonCmd.Env = append(daemonCmd.Env, fmt.Sprintf(\"IPFS_PATH=%s\", ipfsPath))\n\trequire.Nil(t, daemonCmd.Start())\n\n\tdefer func() {\n\t\trequire.Nil(t, daemonCmd.Process.Kill())\n\t}()\n\n\t// Wait until the daemon actually offers the API interface:\n\tlocalAddr := fmt.Sprintf(\"localhost:%d\", apiPort)\n\tfor tries := 0; tries < 200; tries++ {\n\t\tif shell.NewShell(localAddr).IsUp() {\n\t\t\tbreak\n\t\t}\n\n\t\ttime.Sleep(100 * time.Millisecond)\n\t}\n\n\t// Actually call the test:\n\tfn(t, ipfsPath)\n\n}\n\n// WithDoubleIpfs starts two IPFS instances in parallel.\nfunc WithDoubleIpfs(t *testing.T, portOff int, fn func(t *testing.T, ipfsPathA, ipfsPathB string)) {\n\tchPathA := make(chan string)\n\tchPathB := make(chan string)\n\tstop := make(chan bool, 2)\n\n\tgo WithIpfs(t, portOff, func(t *testing.T, ipfsPathA string) {\n\t\tchPathA <- ipfsPathA\n\t\t<-stop\n\t})\n\n\tgo WithIpfs(t, portOff+1, func(t *testing.T, ipfsPathB string) {\n\t\tchPathB <- ipfsPathB\n\t\t<-stop\n\t})\n\n\tfn(t, <-chPathA, <-chPathB)\n\tstop <- true\n\tstop <- true\n}\n" }, { "path": "backend/httpipfs/testing_test.go", "content": "package httpipfs\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestIpfsStartup(t *testing.T) {\n\tt.Skipf(\"will be replaced by bash based e2e tests\")\n\n\tWithIpfs(t, 1, func(t *testing.T, ipfsPath string) {\n\t\tnd, err := NewNode(ipfsPath, \"\")\n\t\trequire.Nil(t, err)\n\n\t\thash, err := nd.Add(bytes.NewReader([]byte(\"hello\")))\n\t\trequire.Nil(t, err, fmt.Sprintf(\"%v\", err))\n\t\trequire.Equal(t, \"QmWfVY9y3xjsixTgbd9AorQxH7VtMpzfx2HaWtsoUYecaX\", hash.String())\n\t})\n}\n\nfunc TestDoubleIpfsStartup(t *testing.T) {\n\tt.Skipf(\"will be replaced by bash based e2e tests\")\n\n\tWithDoubleIpfs(t, 1, func(t *testing.T, ipfsPathA, ipfsPathB string) {\n\t\tndA, err := NewNode(ipfsPathA, \"\")\n\t\trequire.Nil(t, err)\n\n\t\tndB, err := NewNode(ipfsPathB, \"\")\n\t\trequire.Nil(t, err)\n\n\t\tidA, err := ndA.Identity()\n\t\trequire.Nil(t, err, fmt.Sprintf(\"%v\", err))\n\n\t\tidB, err := ndB.Identity()\n\t\trequire.Nil(t, err)\n\n\t\trequire.NotEqual(t, idA.Addr, idB.Addr)\n\t})\n}\n" }, { "path": "backend/httpipfs/version.go", "content": "package httpipfs\n\n// VersionInfo holds version info (yeah, golint)\ntype VersionInfo struct {\n\tsemVer, name, rev string\n}\n\n// SemVer returns a VersionInfo string complying semantic versioning\nfunc (v *VersionInfo) SemVer() string { return v.semVer }\n\n// Name returns the name of the backend\nfunc (v *VersionInfo) Name() string { return v.name }\n\n// Rev returns the git revision of the backend\nfunc (v *VersionInfo) Rev() string { return v.rev }\n\n// Version returns detailed VersionInfo info as struct\nfunc (n *Node) Version() *VersionInfo {\n\tv, rev, err := n.sh.Version()\n\tif err != nil {\n\t\treturn nil\n\t}\n\n\treturn &VersionInfo{\n\t\tsemVer: v,\n\t\tname: \"go-ipfs\",\n\t\trev: rev,\n\t}\n}\n" }, { "path": "backend/mock/mock.go", "content": "package mock\n\nimport (\n\t\"github.com/sahib/brig/catfs\"\n\teventsMock \"github.com/sahib/brig/events/mock\"\n\tnetMock \"github.com/sahib/brig/net/mock\"\n\trepoMock \"github.com/sahib/brig/repo/mock\"\n)\n\n// Backend is used for local testing.\ntype Backend struct {\n\t*catfs.MemFsBackend\n\t*repoMock.RepoBackend\n\t*netMock.NetBackend\n\t*eventsMock.EventsBackend\n}\n\n// NewMockBackend returns a backend.Backend that operates only in memory\n// and does not use any resources outliving the own process, except the net\n// part which stores connection info on disk.\nfunc NewMockBackend(path, owner string) *Backend {\n\treturn &Backend{\n\t\tMemFsBackend: catfs.NewMemFsBackend(),\n\t\tRepoBackend: repoMock.NewMockRepoBackend(),\n\t\tNetBackend: netMock.NewNetBackend(path, owner),\n\t\tEventsBackend: eventsMock.NewEventsBackend(owner),\n\t}\n}\n\n// VersionInfo holds version info (yeah, golint)\ntype VersionInfo struct {\n\tsemVer, name, rev string\n}\n\n// SemVer returns a version string complying semantic versioning\nfunc (v *VersionInfo) SemVer() string { return v.semVer }\n\n// Name returns the name of the backend\nfunc (v *VersionInfo) Name() string { return v.name }\n\n// Rev returns the git revision of the backend\nfunc (v *VersionInfo) Rev() string { return v.rev }\n\n// Version returns detailed version info as struct\nfunc Version() *VersionInfo {\n\treturn &VersionInfo{\n\t\tsemVer: \"0.0.1\",\n\t\tname: \"mock\",\n\t\trev: \"HEAD\",\n\t}\n}\n" }, { "path": "bench/bench.go", "content": "package bench\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"math/rand\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"runtime\"\n\t\"sort\"\n\t\"strings\"\n\t\"syscall\"\n\t\"time\"\n\n\t\"github.com/pkg/xattr\"\n\t\"github.com/sahib/brig/backend/httpipfs\"\n\t\"github.com/sahib/brig/catfs/mio\"\n\t\"github.com/sahib/brig/client\"\n\t\"github.com/sahib/brig/client/clienttest\"\n\t\"github.com/sahib/brig/fuse/fusetest\"\n\t\"github.com/sahib/brig/repo/hints\"\n\t\"github.com/sahib/brig/server\"\n\t\"github.com/sahib/brig/util/testutil\"\n)\n\n// Run is a single benchmark run\ntype Run struct {\n\tTook time.Duration\n\tAllocs int64\n\tCompressionRatio float32\n}\n\n// Runs is a list of individual runs\ntype Runs []Run\n\n// Average returns a fictional average run out of all runs\nfunc (runs Runs) Average() Run {\n\tsum := Run{}\n\tfor _, run := range runs {\n\t\tsum.Took += run.Took\n\t\tsum.Allocs += run.Allocs\n\t\tsum.CompressionRatio += run.CompressionRatio\n\t}\n\n\treturn Run{\n\t\tTook: sum.Took / time.Duration(len(runs)),\n\t\tAllocs: sum.Allocs / int64(len(runs)),\n\t\tCompressionRatio: sum.CompressionRatio / float32(len(runs)),\n\t}\n}\n\n// Bench is the interface every benchmark needs to implement.\ntype Bench interface {\n\t// SupportHints should return true for benchmarks where\n\t// passing hint influences the benchmark result.\n\tSupportHints() bool\n\n\t// CanBeVerified should return true when the test\n\t// can use the verifier (i.e. is a read test)\n\tCanBeVerified() bool\n\n\t// Bench should read the input from `r` and apply `hint` if applicable.\n\t// The time needed to process all of `r` should be returned.\n\tBench(hint hints.Hint, size int64, r io.Reader, w io.Writer) (*Run, error)\n\n\t// Close should clean up the benchmark.\n\tClose() error\n}\n\nvar (\n\tdummyKey = make([]byte, 32)\n)\n\nfunc withRunStats(size int64, fn func() (int64, error)) (*Run, error) {\n\tstart := time.Now()\n\n\tvar memBefore, memAfter runtime.MemStats\n\truntime.ReadMemStats(&memBefore)\n\twritten, err := fn()\n\truntime.ReadMemStats(&memAfter)\n\ttook := time.Since(start)\n\treturn &Run{\n\t\tTook: took,\n\t\tCompressionRatio: float32(written) / float32(size),\n\t\tAllocs: int64(memAfter.Mallocs) - int64(memBefore.Mallocs),\n\t}, err\n}\n\n//////////\n\ntype memcpyBench struct{}\n\nfunc newMemcpyBench(_ string, _ bool) (Bench, error) {\n\treturn memcpyBench{}, nil\n}\n\nfunc (n memcpyBench) SupportHints() bool { return false }\n\nfunc (n memcpyBench) CanBeVerified() bool { return true }\n\nfunc (n memcpyBench) Bench(hint hints.Hint, size int64, r io.Reader, verifier io.Writer) (*Run, error) {\n\t// NOTE: Use DumbCopy, since io.Copy would use the\n\t// ReadFrom of ioutil.Discard. This is lightning fast.\n\t// We want to measure actual time to copy in memory.\n\n\treturn withRunStats(size, func() (int64, error) {\n\t\treturn testutil.DumbCopy(verifier, r, false, false)\n\t})\n}\n\nfunc (n memcpyBench) Close() error { return nil }\n\n//////////\n\ntype serverCommon struct {\n\tdaemon *server.Server\n\tclient *client.Client\n}\n\nfunc newServerCommon(ipfsPath string) (*serverCommon, error) {\n\tbackendName := \"mock\"\n\tif ipfsPath != \"\" {\n\t\tbackendName = \"httpipfs\"\n\t}\n\n\tsrv, err := clienttest.StartDaemon(\"ali\", backendName, ipfsPath)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tctl, err := client.Dial(context.Background(), srv.DaemonURL())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &serverCommon{\n\t\tdaemon: srv,\n\t\tclient: ctl,\n\t}, nil\n}\n\nfunc (sc *serverCommon) Close() error {\n\tsc.daemon.Close()\n\tsc.client.Close()\n\treturn nil\n}\n\ntype serverStageBench struct {\n\tcommon *serverCommon\n}\n\nfunc newServerStageBench(ipfsPath string, _ bool) (Bench, error) {\n\tcommon, err := newServerCommon(ipfsPath)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &serverStageBench{common: common}, nil\n}\n\nfunc (s *serverStageBench) SupportHints() bool { return true }\n\nfunc (s *serverStageBench) CanBeVerified() bool { return false }\n\nfunc (s *serverStageBench) Bench(hint hints.Hint, size int64, r io.Reader, verifier io.Writer) (*Run, error) {\n\tpath := fmt.Sprintf(\"/path_%d\", rand.Int31())\n\n\tc := string(hint.CompressionAlgo)\n\te := string(hint.EncryptionAlgo)\n\tif err := s.common.client.HintSet(path, &c, &e); err != nil {\n\t\treturn nil, err\n\t}\n\n\t// That's just for cleaning up after each test.\n\tdefer s.common.client.Remove(path)\n\n\treturn withRunStats(size, func() (int64, error) {\n\t\treturn size, s.common.client.StageFromReader(path, r)\n\t})\n}\n\nfunc (s *serverStageBench) Close() error {\n\treturn s.common.Close()\n}\n\ntype serverCatBench struct {\n\tcommon *serverCommon\n}\n\nfunc newServerCatBench(ipfsPath string, _ bool) (Bench, error) {\n\tcommon, err := newServerCommon(ipfsPath)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &serverCatBench{common: common}, nil\n}\n\nfunc (s *serverCatBench) SupportHints() bool { return true }\n\nfunc (s *serverCatBench) CanBeVerified() bool { return true }\n\nfunc (s *serverCatBench) Bench(hint hints.Hint, size int64, r io.Reader, verifier io.Writer) (*Run, error) {\n\tpath := fmt.Sprintf(\"/path_%d\", rand.Int31())\n\tc := string(hint.CompressionAlgo)\n\te := string(hint.EncryptionAlgo)\n\n\tif err := s.common.client.HintSet(path, &c, &e); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := s.common.client.StageFromReader(path, r); err != nil {\n\t\treturn nil, err\n\t}\n\n\t// That's just for cleaning up after each test.\n\tdefer s.common.client.Remove(path)\n\n\treturn withRunStats(size, func() (int64, error) {\n\t\tstream, err := s.common.client.Cat(path, true)\n\t\tif err != nil {\n\t\t\treturn 0, err\n\t\t}\n\n\t\tdefer stream.Close()\n\t\treturn testutil.DumbCopy(verifier, stream, false, false)\n\t})\n}\n\nfunc (s *serverCatBench) Close() error {\n\treturn s.common.Close()\n}\n\n//////////\n\ntype mioWriterBench struct{}\n\nfunc newMioWriterBench(_ string, _ bool) (Bench, error) {\n\treturn &mioWriterBench{}, nil\n}\n\nfunc (m *mioWriterBench) SupportHints() bool { return true }\n\nfunc (m *mioWriterBench) CanBeVerified() bool { return false }\n\nfunc (m *mioWriterBench) Bench(hint hints.Hint, size int64, r io.Reader, verifier io.Writer) (*Run, error) {\n\tstream, _, err := mio.NewInStream(r, \"\", dummyKey, hint)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn withRunStats(size, func() (int64, error) {\n\t\tdefer stream.Close()\n\t\treturn testutil.DumbCopy(ioutil.Discard, stream, false, false)\n\t})\n}\n\nfunc (m *mioWriterBench) Close() error {\n\treturn nil\n}\n\n//////////\n\ntype mioReaderBench struct{}\n\nfunc newMioReaderBench(_ string, _ bool) (Bench, error) {\n\treturn &mioReaderBench{}, nil\n}\n\nfunc (m *mioReaderBench) SupportHints() bool { return true }\n\nfunc (m *mioReaderBench) CanBeVerified() bool { return true }\n\nfunc (m *mioReaderBench) Bench(hint hints.Hint, size int64, r io.Reader, verifier io.Writer) (*Run, error) {\n\t// Produce a buffer with encoded data in the right size.\n\t// This is not benched, only the reading of it is.\n\tinStream, _, err := mio.NewInStream(r, \"\", dummyKey, hint)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdefer inStream.Close()\n\n\t// Read it to memory before measuring.\n\t// We do not want to count the encoding in the bench time.\n\tstreamData, err := ioutil.ReadAll(inStream)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn withRunStats(size, func() (int64, error) {\n\t\toutStream, err := mio.NewOutStream(\n\t\t\tbytes.NewReader(streamData),\n\t\t\thint.IsRaw(),\n\t\t\tdummyKey,\n\t\t)\n\n\t\tif err != nil {\n\t\t\treturn -1, err\n\t\t}\n\n\t\tdefer outStream.Close()\n\n\t\treturn testutil.DumbCopy(verifier, outStream, false, false)\n\t})\n}\n\nfunc (m *mioReaderBench) Close() error {\n\treturn nil\n}\n\n//////////\n\ntype ipfsAddOrCatBench struct {\n\tipfsPath string\n\tisAdd bool\n}\n\nfunc newIPFSAddBench(ipfsPath string, isAdd bool) (Bench, error) {\n\treturn &ipfsAddOrCatBench{ipfsPath: ipfsPath, isAdd: isAdd}, nil\n}\n\nfunc (ia *ipfsAddOrCatBench) SupportHints() bool { return false }\n\nfunc (ia *ipfsAddOrCatBench) CanBeVerified() bool { return !ia.isAdd }\n\nfunc (ia *ipfsAddOrCatBench) Bench(hint hints.Hint, size int64, r io.Reader, verifier io.Writer) (*Run, error) {\n\tnd, err := httpipfs.NewNode(ia.ipfsPath, \"\")\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdefer nd.Close()\n\n\tif ia.isAdd {\n\t\treturn withRunStats(size, func() (int64, error) {\n\t\t\t_, err := nd.Add(r)\n\t\t\treturn size, err\n\t\t})\n\t}\n\n\thash, err := nd.Add(r)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn withRunStats(size, func() (int64, error) {\n\t\tstream, err := nd.Cat(hash)\n\t\tif err != nil {\n\t\t\treturn -1, err\n\t\t}\n\n\t\treturn testutil.DumbCopy(verifier, stream, false, false)\n\t})\n}\n\nfunc (ia *ipfsAddOrCatBench) Close() error {\n\treturn nil\n}\n\n//////////\n\ntype fuseWriteOrReadBench struct {\n\tipfsPath string\n\tisWrite bool\n\n\ttmpDir string\n\tctl *fusetest.Client\n\tproc *os.Process\n}\n\nfunc newFuseWriteOrReadBench(ipfsPath string, isWrite bool) (Bench, error) {\n\ttmpDir, err := ioutil.TempDir(\"\", \"brig-fuse-bench-*\")\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tunixSocket := \"unix:\" + filepath.Join(tmpDir, \"socket\")\n\n\tproc, err := fusetest.LaunchAsProcess(fusetest.Options{\n\t\tMountPath: filepath.Join(tmpDir, \"mount\"),\n\t\tCatfsPath: filepath.Join(tmpDir, \"catfs\"),\n\t\tIpfsPathOrMultiaddr: ipfsPath,\n\t\tURL: unixSocket,\n\t})\n\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// bit time to start things up:\n\ttime.Sleep(500 * time.Millisecond)\n\n\tctl, err := fusetest.Dial(unixSocket)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &fuseWriteOrReadBench{\n\t\tipfsPath: ipfsPath,\n\t\tisWrite: isWrite,\n\t\ttmpDir: tmpDir,\n\t\tproc: proc,\n\t\tctl: ctl,\n\t}, nil\n}\n\nfunc (fb *fuseWriteOrReadBench) SupportHints() bool { return true }\n\nfunc (fb *fuseWriteOrReadBench) CanBeVerified() bool { return !fb.isWrite }\n\nfunc (fb *fuseWriteOrReadBench) Bench(hint hints.Hint, size int64, r io.Reader, verifier io.Writer) (*Run, error) {\n\tmountDir := filepath.Join(fb.tmpDir, \"mount\")\n\ttestPath := filepath.Join(mountDir, fmt.Sprintf(\"/path_%d\", rand.Int31()))\n\n\tconst (\n\t\txattrEnc = \"user.brig.hints.encryption\"\n\t\txattrZip = \"user.brig.hints.compression\"\n\t)\n\n\t// Make sure hints are followed:\n\tif err := xattr.Set(mountDir, xattrEnc, []byte(hint.EncryptionAlgo)); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := xattr.Set(mountDir, xattrZip, []byte(hint.CompressionAlgo)); err != nil {\n\t\treturn nil, err\n\t}\n\n\ttook, err := withRunStats(size, func() (int64, error) {\n\t\tfd, err := os.OpenFile(testPath, os.O_CREATE|os.O_WRONLY, 0600)\n\t\tif err != nil {\n\t\t\treturn -1, err\n\t\t}\n\n\t\tdefer fd.Close()\n\n\t\treturn testutil.DumbCopy(fd, r, false, false)\n\t})\n\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif fb.isWrite {\n\t\t// test is done already, no need to read-back.\n\t\treturn took, nil\n\t}\n\n\ttook, err = withRunStats(size, func() (int64, error) {\n\t\t// NOTE: We have to use syscall.O_DIRECT here in order to\n\t\t// bypass the kernel page cache. The write above fills it with\n\t\t// data immediately, thus this read can yield 10x times higher\n\t\t// results (which you still might get in practice, if lucky)\n\t\tfd, err := os.OpenFile(testPath, os.O_RDONLY|syscall.O_DIRECT, 0600)\n\t\tif err != nil {\n\t\t\treturn -1, err\n\t\t}\n\n\t\tdefer fd.Close()\n\n\t\treturn testutil.DumbCopy(verifier, fd, false, false)\n\t})\n\n\treturn took, err\n}\n\nfunc (fb *fuseWriteOrReadBench) Close() error {\n\tfb.ctl.QuitServer()\n\ttime.Sleep(2 * time.Second)\n\tfb.proc.Signal(syscall.SIGTERM)\n\n\tvar lastError error\n\tfor retries := 0; retries < 10; retries++ {\n\t\tif err := os.RemoveAll(fb.tmpDir); err != nil {\n\t\t\ttime.Sleep(200 * time.Millisecond)\n\t\t\tlastError = err\n\t\t\tcontinue\n\t\t}\n\n\t\tlastError = nil\n\t\tbreak\n\t}\n\n\treturn lastError\n}\n\n//////////\n\nvar (\n\t// Convention:\n\t// - If it's using ipfs, put it in the name.\n\t// - If it's writing things, put that in the name too as \"write\".\n\tbenchMap = map[string]func(string, bool) (Bench, error){\n\t\t\"memcpy\": newMemcpyBench,\n\t\t\"brig-write-mem\": newServerStageBench,\n\t\t\"brig-read-mem\": newServerCatBench,\n\t\t\"brig-write-ipfs\": newServerStageBench,\n\t\t\"brig-read-ipfs\": newServerCatBench,\n\t\t\"mio-write\": newMioWriterBench,\n\t\t\"mio-read\": newMioReaderBench,\n\t\t\"ipfs-write\": newIPFSAddBench,\n\t\t\"ipfs-read\": newIPFSAddBench,\n\t\t\"fuse-write-mem\": newFuseWriteOrReadBench,\n\t\t\"fuse-write-ipfs\": newFuseWriteOrReadBench,\n\t\t\"fuse-read-mem\": newFuseWriteOrReadBench,\n\t\t\"fuse-read-ipfs\": newFuseWriteOrReadBench,\n\t}\n)\n\n// ByName returns the benchmark with this name, or an error\n// if none. If IPFS is used, it should be given as `ipfsPath`.\nfunc ByName(name, ipfsPath string) (Bench, error) {\n\tnewBench, ok := benchMap[name]\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"no such bench: %s\", name)\n\t}\n\n\treturn newBench(ipfsPath, strings.Contains(name, \"write\"))\n}\n\n// BenchmarkNames returns all possible benchmark names\n// in an defined & stable sorting.\nfunc BenchmarkNames() []string {\n\tnames := []string{}\n\tfor name := range benchMap {\n\t\tnames = append(names, name)\n\t}\n\n\tsort.Slice(names, func(i, j int) bool {\n\t\tif names[i] == names[j] {\n\t\t\treturn false\n\t\t}\n\n\t\tspecials := []string{\n\t\t\t\"memcpy\",\n\t\t\t\"mio\",\n\t\t}\n\n\t\tfor _, special := range specials {\n\t\t\tv := strings.HasSuffix(names[i], special)\n\t\t\tif v || strings.HasSuffix(names[j], special) {\n\t\t\t\treturn v\n\t\t\t}\n\t\t}\n\n\t\treturn names[i] < names[j]\n\t})\n\n\treturn names\n}\n" }, { "path": "bench/inputs.go", "content": "package bench\n\nimport (\n\t\"bytes\"\n\t\"encoding/binary\"\n\t\"fmt\"\n\t\"io\"\n\t\"sort\"\n\n\t\"github.com/sahib/brig/util/testutil\"\n)\n\n// Verifier is a io.Writer that should be used for benchmarks\n// that read encoded data. It verifies that the data is actually\n// correct in the sense that it is equal to the original input.\ntype Verifier interface {\n\tio.Writer\n\n\t// MissingBytes returns the diff of bytes to the original input.\n\t// This number can be negative when too much data was written.\n\t// Only 0 is a valid value after the benchmark finished.\n\tMissingBytes() int64\n}\n\n// Input generates input for a benchmark. It defines how the data looks that\n// is fed to the streaming system.\ntype Input interface {\n\tReader(seed uint64) (io.Reader, error)\n\tSize() int64\n\tVerifier() (Verifier, error)\n\tClose() error\n}\n\nfunc benchData(size uint64, name string) []byte {\n\tswitch name {\n\tcase \"random\":\n\t\treturn testutil.CreateRandomDummyBuf(int64(size), 23)\n\tcase \"ten\":\n\t\treturn testutil.CreateDummyBuf(int64(size))\n\tcase \"mixed\":\n\t\treturn testutil.CreateMixedDummyBuf(int64(size), 42)\n\tdefault:\n\t\treturn nil\n\t}\n}\n\n//////////\n\ntype memVerifier struct {\n\texpect []byte\n\tcounter int64\n}\n\nfunc (m *memVerifier) Write(buf []byte) (int, error) {\n\tif int64(len(buf))+m.counter > int64(len(m.expect)) {\n\t\treturn -1, fmt.Errorf(\"verify: got too much data\")\n\t}\n\n\tslice := m.expect[m.counter : m.counter+int64(len(buf))]\n\tif !bytes.Equal(slice, buf) {\n\t\treturn -1, fmt.Errorf(\"verify: data differs in block at %d\", m.counter)\n\t}\n\n\tm.counter += int64(len(buf))\n\n\t// Just nod off the data and let GC do the rest.\n\treturn len(buf), nil\n}\n\nfunc (m *memVerifier) MissingBytes() int64 {\n\treturn int64(len(m.expect)) - m.counter\n}\n\ntype memInput struct {\n\tbuf []byte\n}\n\nfunc newMemInput(size uint64, name string) Input {\n\treturn &memInput{buf: benchData(size, name)}\n}\n\nfunc (ni *memInput) Reader(seed uint64) (io.Reader, error) {\n\t// Put a few bytes difference at the start to make the complete\n\t// stream different than the last seed. This is here to avoid\n\t// that consequent runs of a benchmark get speed ups because\n\t// they can cache inputs.\n\tbinary.LittleEndian.PutUint64(ni.buf, seed)\n\treturn bytes.NewReader(ni.buf), nil\n}\n\nfunc (ni *memInput) Verifier() (Verifier, error) {\n\treturn &memVerifier{\n\t\texpect: ni.buf,\n\t\tcounter: 0,\n\t}, nil\n}\n\nfunc (ni *memInput) Size() int64 {\n\treturn int64(len(ni.buf))\n}\n\nfunc (ni *memInput) Close() error {\n\treturn nil\n}\n\n//////////\n\nvar (\n\tinputMap = map[string]func(size uint64) (Input, error){\n\t\t\"ten\": func(size uint64) (Input, error) {\n\t\t\treturn newMemInput(size, \"ten\"), nil\n\t\t},\n\t\t\"random\": func(size uint64) (Input, error) {\n\t\t\treturn newMemInput(size, \"random\"), nil\n\t\t},\n\t\t\"mixed\": func(size uint64) (Input, error) {\n\t\t\treturn newMemInput(size, \"mixed\"), nil\n\t\t},\n\t}\n)\n\n// InputByName fetches the input by it's name and returns an input\n// that will produce data with `size` bytes.\nfunc InputByName(name string, size uint64) (Input, error) {\n\tnewInput, ok := inputMap[name]\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"no such input: %s\", name)\n\t}\n\n\treturn newInput(size)\n}\n\n// InputNames returns the sorted list of all possible inputs.\nfunc InputNames() []string {\n\tnames := []string{}\n\tfor name := range inputMap {\n\t\tnames = append(names, name)\n\t}\n\n\tsort.Strings(names)\n\treturn names\n}\n" }, { "path": "bench/runner.go", "content": "package bench\n\nimport (\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"os/signal\"\n\t\"runtime\"\n\t\"sort\"\n\t\"strings\"\n\t\"syscall\"\n\t\"time\"\n\n\t\"github.com/sahib/brig/repo/hints\"\n\t\"github.com/sahib/brig/repo/setup\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// Config define how the benchmarks are run.\ntype Config struct {\n\tInputName string `json:\"input_name\"`\n\tBenchName string `json:\"bench_name\"`\n\tSize uint64 `json:\"size\"`\n\tEncryption string `json:\"encryption\"`\n\tCompression string `json:\"compression\"`\n\tSamples int `json:\"samples\"`\n}\n\n// Result is the result of a single benchmark run.\ntype Result struct {\n\tName string `json:\"name\"`\n\tConfig Config `json:\"config\"`\n\tEncryption string `json:\"encryption\"`\n\tCompression string `json:\"compression\"`\n\tTook time.Duration `json:\"took\"`\n\tThroughput float64 `json:\"throughput\"`\n\tCompressionRate float32 `json:\"compression_rate\"`\n\tAllocs int64 `json:\"allocs\"`\n}\n\n// buildHints handles wildcards for compression and/or encryption.\n// If no wildcards are specified, we just take what is set in `cfg`.\nfunc buildHints(cfg Config) []hints.Hint {\n\tencIsWildcard := cfg.Encryption == \"*\"\n\tzipIsWildcard := cfg.Compression == \"*\"\n\n\tif encIsWildcard && zipIsWildcard {\n\t\treturn hints.AllPossibleHints()\n\t}\n\n\tif encIsWildcard {\n\t\ths := []hints.Hint{}\n\t\tfor _, encAlgo := range hints.ValidEncryptionHints() {\n\t\t\ths = append(hs, hints.Hint{\n\t\t\t\tCompressionAlgo: hints.CompressionHint(cfg.Compression),\n\t\t\t\tEncryptionAlgo: hints.EncryptionHint(encAlgo),\n\t\t\t})\n\t\t}\n\n\t\treturn hs\n\t}\n\n\tif zipIsWildcard {\n\t\ths := []hints.Hint{}\n\t\tfor _, zipAlgo := range hints.ValidCompressionHints() {\n\t\t\ths = append(hs, hints.Hint{\n\t\t\t\tCompressionAlgo: hints.CompressionHint(zipAlgo),\n\t\t\t\tEncryptionAlgo: hints.EncryptionHint(cfg.Encryption),\n\t\t\t})\n\t\t}\n\n\t\treturn hs\n\t}\n\n\treturn []hints.Hint{{\n\t\tCompressionAlgo: hints.CompressionHint(cfg.Compression),\n\t\tEncryptionAlgo: hints.EncryptionHint(cfg.Encryption),\n\t}}\n}\n\nfunc sortHints(hs []hints.Hint) []hints.Hint {\n\tsort.Slice(hs, func(i, j int) bool {\n\t\treturn hs[i].Less(hs[j])\n\t})\n\n\t// sorts in-place, but also return for ease of use.\n\treturn hs\n}\n\nfunc benchmarkSingle(cfg Config, fn func(result Result), ipfsPath string) error {\n\tin, err := InputByName(cfg.InputName, cfg.Size)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdefer in.Close()\n\n\tout, err := ByName(cfg.BenchName, ipfsPath)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdefer out.Close()\n\n\tsigCh := make(chan os.Signal, 1)\n\tsignal.Notify(sigCh, syscall.SIGINT)\n\tdefer signal.Stop(sigCh)\n\n\tfor _, hint := range sortHints(buildHints(cfg)) {\n\t\tselect {\n\t\tcase <-sigCh:\n\t\t\tfmt.Println(\"Interrupted\")\n\t\t\treturn nil\n\t\tdefault:\n\t\t\t// just continue\n\t\t}\n\n\t\tsupportsHints := out.SupportHints()\n\t\tif !supportsHints {\n\t\t\t// Indicate in output that nothing was encrypted or compressed.\n\t\t\thint.CompressionAlgo = hints.CompressionNone\n\t\t\thint.EncryptionAlgo = hints.EncryptionNone\n\t\t}\n\n\t\tif hint.CompressionAlgo == hints.CompressionGuess {\n\t\t\t// NOTE: We do not benchmark guessing here.\n\t\t\t// Simply reason is that we do not know from the output\n\t\t\t// which algorithm was actually used.\n\t\t\tcontinue\n\t\t}\n\n\t\tvar runs Runs\n\n\t\t// probably doesn't do much, just to clean any leftover memory.\n\t\truntime.GC()\n\n\t\tfor seed := uint64(0); seed < uint64(cfg.Samples); seed++ {\n\t\t\tr, err := in.Reader(seed)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tv, err := in.Verifier()\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\trun, err := out.Bench(hint, in.Size(), r, v)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\truns = append(runs, *run)\n\n\t\t\t// Most write-only benchmarks cannot be verified, since\n\t\t\t// we modify the stream and the verifier checks that the stream\n\t\t\t// is equal to the input. Most read tests involve the same logic\n\t\t\t// as writing though, so the writer has to work for that.\n\t\t\tif out.CanBeVerified() {\n\t\t\t\tif missing := v.MissingBytes(); missing != 0 {\n\t\t\t\t\tlog.Warnf(\"not all or too much data received in verify: %d\", missing)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\tavgRun := runs.Average()\n\t\tthroughput := (float64(cfg.Size) / 1000 / 1000) / (float64(avgRun.Took) / float64(time.Second))\n\t\tfn(Result{\n\t\t\tName: fmt.Sprintf(\"%s:%s_%s\", cfg.BenchName, cfg.InputName, hint),\n\t\t\tEncryption: string(hint.EncryptionAlgo),\n\t\t\tCompression: string(hint.CompressionAlgo),\n\t\t\tConfig: cfg,\n\t\t\tTook: avgRun.Took,\n\t\t\tThroughput: throughput,\n\t\t\tCompressionRate: avgRun.CompressionRatio,\n\t\t\tAllocs: avgRun.Allocs,\n\t\t})\n\n\t\tif !supportsHints {\n\t\t\t// If there are no hints there is no point.\n\t\t\t// of repeating the benchmark several times.\n\t\t\tbreak\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// IPFS is expensive to set-up, so let's do it only once.\nfunc ipfsIsNeeded(cfgs []Config) bool {\n\tfor _, cfg := range cfgs {\n\t\tif strings.Contains(strings.ToLower(cfg.BenchName), \"ipfs\") {\n\t\t\treturn true\n\t\t}\n\t}\n\n\treturn false\n}\n\n// Benchmark runs the benchmarks specified by `cfgs` and call `fn` on each result.\nfunc Benchmark(cfgs []Config, fn func(result Result)) error {\n\tneedsIPFS := ipfsIsNeeded(cfgs)\n\tvar result *setup.Result\n\n\tif needsIPFS {\n\t\tvar err error\n\t\tlog.Infof(\"Setting up IPFS for the benchmarks...\")\n\n\t\tipfsPath, err := ioutil.TempDir(\"\", \"brig-iobench-ipfs-repo-*\")\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tresult, err = setup.IPFS(setup.Options{\n\t\t\tLogWriter: ioutil.Discard,\n\t\t\tSetup: true,\n\t\t\tSetDefaultConfig: true,\n\t\t\tSetExtraConfig: true,\n\t\t\tIpfsPath: ipfsPath,\n\t\t\tInitProfile: \"test\",\n\t\t})\n\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tfor _, cfg := range cfgs {\n\t\tvar ipfsPath string\n\t\tif result != nil {\n\t\t\tipfsPath = result.IpfsPath\n\t\t}\n\n\t\tif err := benchmarkSingle(cfg, fn, ipfsPath); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tif needsIPFS {\n\t\tif result.IpfsPath != \"\" {\n\t\t\tos.RemoveAll(result.IpfsPath)\n\t\t}\n\n\t\tif result.PID > 0 {\n\t\t\tproc, err := os.FindProcess(result.PID)\n\t\t\tif err != nil {\n\t\t\t\tlog.WithError(err).Warnf(\"failed to get IPFS PID\")\n\t\t\t} else {\n\t\t\t\tif err := proc.Kill(); err != nil {\n\t\t\t\t\tlog.WithError(err).Warnf(\"failed to kill IPFS PID\")\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\treturn nil\n}\n" }, { "path": "bench/stats.go", "content": "package bench\n\nimport (\n\t\"time\"\n\n\t\"github.com/klauspost/cpuid/v2\"\n)\n\n// Stats are system statistics that might influence the benchmark result.\ntype Stats struct {\n\tTime time.Time `json:\"time\"`\n\tCPUBrandName string `json:\"cpu_brand_name\"`\n\tLogicalCores int `json:\"logical_cores\"`\n\tHasAESNI bool `json:\"has_aesni\"`\n}\n\n// FetchStats returns the current statistics.\nfunc FetchStats() Stats {\n\treturn Stats{\n\t\tTime: time.Now(),\n\t\tCPUBrandName: cpuid.CPU.BrandName,\n\t\tLogicalCores: cpuid.CPU.LogicalCores,\n\t\tHasAESNI: cpuid.CPU.Supports(cpuid.AESNI),\n\t}\n}\n" }, { "path": "brig.go", "content": "package main\n\nimport (\n\t\"os\"\n\n\t\"github.com/sahib/brig/cmd\"\n)\n\nfunc main() {\n\tos.Exit(cmd.RunCmdline(os.Args))\n}\n" }, { "path": "catfs/backend.go", "content": "package catfs\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\n\t\"github.com/sahib/brig/catfs/mio\"\n\t\"github.com/sahib/brig/catfs/mio/chunkbuf\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\t\"github.com/sahib/brig/util/testutil\"\n)\n\n// ErrNoSuchHash should be returned whenever the backend is unable\n// to find an object referenced to by this hash.\ntype ErrNoSuchHash struct {\n\twhat h.Hash\n}\n\nfunc (eh ErrNoSuchHash) Error() string {\n\treturn fmt.Sprintf(\"No such hash: %s\", eh.what.B58String())\n}\n\n// FsBackend is the interface that needs to be implemented by the data\n// management layer.\ntype FsBackend interface {\n\t// Cat should find the object referenced to by `hash` and\n\t// make its data available as mio.Stream.\n\tCat(hash h.Hash) (mio.Stream, error)\n\n\t// Add should read all data in `r` and return the hash under\n\t// which it can be accessed on later.\n\tAdd(r io.Reader) (h.Hash, error)\n\n\t// Pin gives the object at `hash` a \"pin\".\n\t// (i.e. it marks the file to be stored indefinitely in local storage)\n\t// When pinning an explicit pin with an implicit pin, the explicit pin\n\t// will stay. Upgrading from implicit to explicit is possible though.\n\tPin(hash h.Hash) error\n\n\t// Unpin removes a previously added pin.\n\t// If an object is already unpinned this is a no op.\n\tUnpin(hash h.Hash) error\n\n\t// IsPinned checks if the file is pinned.\n\tIsPinned(hash h.Hash) (bool, error)\n\n\t// IsCached checks if the file contents are available locally.\n\tIsCached(hash h.Hash) (bool, error)\n\n\t// CachedSize returns the backend size for a given hash\n\t// Nehative indicates that cachedSize is unknown\n\tCachedSize(hash h.Hash) (int64, error)\n}\n\n// MemFsBackend is a mock structure that implements FsBackend.\ntype MemFsBackend struct {\n\tdata map[string][]byte\n\tpins map[string]bool\n}\n\n// NewMemFsBackend returns a MemFsBackend (useful for writing tests)\nfunc NewMemFsBackend() *MemFsBackend {\n\treturn &MemFsBackend{\n\t\tdata: make(map[string][]byte),\n\t\tpins: make(map[string]bool),\n\t}\n}\n\n// Cat implements FsBackend.Cat by querying memory.\nfunc (mb *MemFsBackend) Cat(hash h.Hash) (mio.Stream, error) {\n\tdata, ok := mb.data[hash.B58String()]\n\tif !ok {\n\t\treturn nil, ErrNoSuchHash{hash}\n\t}\n\n\tchunkBuf := chunkbuf.NewChunkBuffer(data)\n\trandRead := testutil.RandomizeReads(chunkBuf, 512, true)\n\n\treturn struct {\n\t\tio.Reader\n\t\tio.Seeker\n\t\tio.Closer\n\t\tio.WriterTo\n\t}{\n\t\tReader: randRead,\n\t\tSeeker: chunkBuf,\n\t\tWriterTo: chunkBuf,\n\t\tCloser: ioutil.NopCloser(chunkBuf),\n\t}, nil\n}\n\n// Add implements FsBackend.Add by storing the data in memory.\nfunc (mb *MemFsBackend) Add(r io.Reader) (h.Hash, error) {\n\tdata, err := ioutil.ReadAll(r)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\thash := h.SumWithBackendHash(data)\n\tmb.data[hash.B58String()] = data\n\treturn hash, nil\n}\n\n// Pin implements FsBackend.Pin by storing a marker in memory.\nfunc (mb *MemFsBackend) Pin(hash h.Hash) error {\n\tmb.pins[hash.B58String()] = true\n\treturn nil\n}\n\n// Unpin implements FsBackend.Unpin by removing a marker in memory.\nfunc (mb *MemFsBackend) Unpin(hash h.Hash) error {\n\tmb.pins[hash.B58String()] = false\n\treturn nil\n}\n\n// IsPinned implements FsBackend.IsPinned by querying a marker in memory.\nfunc (mb *MemFsBackend) IsPinned(hash h.Hash) (bool, error) {\n\tisPinned, ok := mb.pins[hash.B58String()]\n\tif !ok {\n\t\treturn false, nil\n\t}\n\n\treturn isPinned, nil\n}\n\n// IsCached implements FsBackend.IsCached by checking if the file exists.\n// If hash found, the file is always cached.\nfunc (mb *MemFsBackend) IsCached(hash h.Hash) (bool, error) {\n\t_, ok := mb.data[hash.B58String()]\n\treturn ok, nil\n}\n\n// CachedSize implements FsBackend.CachedSize by returnig data size\n// If hash found, the file is always cached.\nfunc (mb *MemFsBackend) CachedSize(hash h.Hash) (int64, error) {\n\tdata, ok := mb.data[hash.B58String()]\n\tif !ok {\n\t\treturn -1, nil // negative indicates unknown size\n\t}\n\treturn int64(len(data)), nil\n}\n" }, { "path": "catfs/capnp/pinner.capnp", "content": "using Go = import \"/go.capnp\";\n\n@0xba762188b0a6e4cf;\n\n$Go.package(\"capnp\");\n$Go.import(\"github.com/sahib/brig/catfs/capnp\");\n\n\nstruct Pin {\n inode @0 :UInt64;\n isPinned @1 :Bool;\n}\n\nstruct PinEntry $Go.doc(\"A single entry for a certain content node\") {\n # Following attributes will be part of the hash:\n pins @0 :List(Pin);\n}\n" }, { "path": "catfs/capnp/pinner.capnp.go", "content": "// Code generated by capnpc-go. DO NOT EDIT.\n\npackage capnp\n\nimport (\n\tcapnp \"zombiezen.com/go/capnproto2\"\n\ttext \"zombiezen.com/go/capnproto2/encoding/text\"\n\tschemas \"zombiezen.com/go/capnproto2/schemas\"\n)\n\ntype Pin struct{ capnp.Struct }\n\n// Pin_TypeID is the unique identifier for the type Pin.\nconst Pin_TypeID = 0x985d53e01674ee95\n\nfunc NewPin(s *capnp.Segment) (Pin, error) {\n\tst, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 16, PointerCount: 0})\n\treturn Pin{st}, err\n}\n\nfunc NewRootPin(s *capnp.Segment) (Pin, error) {\n\tst, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 16, PointerCount: 0})\n\treturn Pin{st}, err\n}\n\nfunc ReadRootPin(msg *capnp.Message) (Pin, error) {\n\troot, err := msg.RootPtr()\n\treturn Pin{root.Struct()}, err\n}\n\nfunc (s Pin) String() string {\n\tstr, _ := text.Marshal(0x985d53e01674ee95, s.Struct)\n\treturn str\n}\n\nfunc (s Pin) Inode() uint64 {\n\treturn s.Struct.Uint64(0)\n}\n\nfunc (s Pin) SetInode(v uint64) {\n\ts.Struct.SetUint64(0, v)\n}\n\nfunc (s Pin) IsPinned() bool {\n\treturn s.Struct.Bit(64)\n}\n\nfunc (s Pin) SetIsPinned(v bool) {\n\ts.Struct.SetBit(64, v)\n}\n\n// Pin_List is a list of Pin.\ntype Pin_List struct{ capnp.List }\n\n// NewPin creates a new list of Pin.\nfunc NewPin_List(s *capnp.Segment, sz int32) (Pin_List, error) {\n\tl, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 16, PointerCount: 0}, sz)\n\treturn Pin_List{l}, err\n}\n\nfunc (s Pin_List) At(i int) Pin { return Pin{s.List.Struct(i)} }\n\nfunc (s Pin_List) Set(i int, v Pin) error { return s.List.SetStruct(i, v.Struct) }\n\nfunc (s Pin_List) String() string {\n\tstr, _ := text.MarshalList(0x985d53e01674ee95, s.List)\n\treturn str\n}\n\n// Pin_Promise is a wrapper for a Pin promised by a client call.\ntype Pin_Promise struct{ *capnp.Pipeline }\n\nfunc (p Pin_Promise) Struct() (Pin, error) {\n\ts, err := p.Pipeline.Struct()\n\treturn Pin{s}, err\n}\n\n// A single entry for a certain content node\ntype PinEntry struct{ capnp.Struct }\n\n// PinEntry_TypeID is the unique identifier for the type PinEntry.\nconst PinEntry_TypeID = 0xdb74f7cf7bc815c6\n\nfunc NewPinEntry(s *capnp.Segment) (PinEntry, error) {\n\tst, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 1})\n\treturn PinEntry{st}, err\n}\n\nfunc NewRootPinEntry(s *capnp.Segment) (PinEntry, error) {\n\tst, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 1})\n\treturn PinEntry{st}, err\n}\n\nfunc ReadRootPinEntry(msg *capnp.Message) (PinEntry, error) {\n\troot, err := msg.RootPtr()\n\treturn PinEntry{root.Struct()}, err\n}\n\nfunc (s PinEntry) String() string {\n\tstr, _ := text.Marshal(0xdb74f7cf7bc815c6, s.Struct)\n\treturn str\n}\n\nfunc (s PinEntry) Pins() (Pin_List, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn Pin_List{List: p.List()}, err\n}\n\nfunc (s PinEntry) HasPins() bool {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s PinEntry) SetPins(v Pin_List) error {\n\treturn s.Struct.SetPtr(0, v.List.ToPtr())\n}\n\n// NewPins sets the pins field to a newly\n// allocated Pin_List, preferring placement in s's segment.\nfunc (s PinEntry) NewPins(n int32) (Pin_List, error) {\n\tl, err := NewPin_List(s.Struct.Segment(), n)\n\tif err != nil {\n\t\treturn Pin_List{}, err\n\t}\n\terr = s.Struct.SetPtr(0, l.List.ToPtr())\n\treturn l, err\n}\n\n// PinEntry_List is a list of PinEntry.\ntype PinEntry_List struct{ capnp.List }\n\n// NewPinEntry creates a new list of PinEntry.\nfunc NewPinEntry_List(s *capnp.Segment, sz int32) (PinEntry_List, error) {\n\tl, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 0, PointerCount: 1}, sz)\n\treturn PinEntry_List{l}, err\n}\n\nfunc (s PinEntry_List) At(i int) PinEntry { return PinEntry{s.List.Struct(i)} }\n\nfunc (s PinEntry_List) Set(i int, v PinEntry) error { return s.List.SetStruct(i, v.Struct) }\n\nfunc (s PinEntry_List) String() string {\n\tstr, _ := text.MarshalList(0xdb74f7cf7bc815c6, s.List)\n\treturn str\n}\n\n// PinEntry_Promise is a wrapper for a PinEntry promised by a client call.\ntype PinEntry_Promise struct{ *capnp.Pipeline }\n\nfunc (p PinEntry_Promise) Struct() (PinEntry, error) {\n\ts, err := p.Pipeline.Struct()\n\treturn PinEntry{s}, err\n}\n\nconst schema_ba762188b0a6e4cf = \"x\\xda\\\\\\xd0\\xb1K\\xebP\\x14\\x06\\xf0\\xef\\xbbI_[\" +\n\t\"x\\xef\\xb5WT\\xe8\\xd4\\x08]\\x14\\xb5\\xd6I\\\\l\\x07\" +\n\t\"\\x07\\x05!Wg\\x85\\x90\\xa6% 7!\\xb9(\\xc5\\x7f\" +\n\t\"@\\\\Eps\\x13\\xdc\\xdc\\xa4\\x82\\xa3\\xe2\\xd6\\xcd\\xc5\\xc5\" +\n\t\"\\xc1I\\xd0\\xd51\\x92\\xc5\\x8a\\xd3\\x81\\x8f\\xc3\\xf9\\x1d\\xbej\" +\n\t\"\\xbf-d\\xe1\\x06P\\xa5\\xc2\\x9f\\xec\\xec\\xc3L\\xbf\\xec\\xec\" +\n\t\"\\x9eC\\xd5(\\xb2\\xd1\\xeb\\xe5\\xf5\\xf1\\xcc\\xc1-\\xec\\\" \" +\n\t\";or+\\x9f\\x1b\\x87`\\xf60\\xf5x4\\xfa4\\xcf\" +\n\t\"\\x905\\x8e\\xf7\\x0a,\\x02\\xad\\xab\\x09\\xcaaQ\\x0e\\xeb\\xf2\" +\n\t\"}\\x0d\\xcc|\\xcf\\xf4\\xd2\\xa6\\xef\\x89X\\xc7\\xcd8\\xd4:\" +\n\t\"H\\x16}/\\xd6qe\\xd5\\x0d\\xb5K\\xaa\\x92e\\x036\" +\n\t\"\\x019\\xbb\\x0c\\xa8\\x86E\\xb5$(\\xd9\\x9ed\\x1e.l\" +\n\t\"\\x02j\\xde\\xa2Z\\x11\\xac\\x87:\\xea\\x06,C\\xb0\\x0cf\" +\n\t\"a\\xea\\xe6\\x07\\xbb\\x00H\\x08\\xf2\\x87g\\xfd\\xf6rn]\" +\n\t\"\\x9b\\x84\\x83\\x1c\\xb5)\\xb2\\xbd\\xd3\\x0bu\\xf7tr\\x0fe\" +\n\t\"\\x0bv\\x1a\\xe4_\\xa0\\xc5mf\\x1d'\\x0du\\x7f?\\xb0\" +\n\t\"\\x9d@\\x9bd\\xe0\\xf4\\xa2\\xc4\\xf1\\x1c?H\\x8c\\x17j\\xc7\" +\n\t\"\\x8f\\xb4\\x09\\xb4qt\\xd4e\\x00(\\xfb\\xfb\\xff\\x7fsy\" +\n\t\"\\x91\\x16UC\\xb0\\x12\\x87:\\xe5\\x7f\\xd0\\xb5\\xc8\\xea\\xb8Z\" +\n\t\"0\\x0f\\xbf\\x02\\x00\\x00\\xff\\xff\\x05\\xdde\\x03\"\n\nfunc init() {\n\tschemas.Register(schema_ba762188b0a6e4cf,\n\t\t0x985d53e01674ee95,\n\t\t0xdb74f7cf7bc815c6)\n}\n" }, { "path": "catfs/core/coreutils.go", "content": "package core\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"path\"\n\t\"strings\"\n\t\"time\"\n\n\te \"github.com/pkg/errors\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\nvar (\n\t// ErrIsGhost is returned by Remove() when calling it on a ghost.\n\tErrIsGhost = errors.New(\"Is a ghost\")\n)\n\n// mkdirParents takes the dirname of repoPath and makes sure all intermediate\n// directories are created. The last directory will be returned.\n\n// If any directory exist already, it will not be touched.\n// You can also think of it as mkdir -p.\nfunc mkdirParents(lkr *Linker, repoPath string) (*n.Directory, error) {\n\trepoPath = path.Clean(repoPath)\n\n\telems := strings.Split(repoPath, \"/\")\n\tfor idx := 0; idx < len(elems)-1; idx++ {\n\t\tdirname := strings.Join(elems[:idx+1], \"/\")\n\t\tif dirname == \"\" {\n\t\t\tdirname = \"/\"\n\t\t}\n\n\t\tdir, err := Mkdir(lkr, dirname, false)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\t// Return it, if it's the last path component:\n\t\tif idx+1 == len(elems)-1 {\n\t\t\treturn dir, nil\n\t\t}\n\t}\n\n\treturn nil, fmt.Errorf(\"Empty path given\")\n}\n\n// Mkdir creates the directory at repoPath and any intermediate directories if\n// createParents is true. It will fail if there is already a file at `repoPath`\n// and it is not a directory.\nfunc Mkdir(lkr *Linker, repoPath string, createParents bool) (dir *n.Directory, err error) {\n\tdirname, basename := path.Split(repoPath)\n\n\t// Take special care of the root node:\n\tif basename == \"\" {\n\t\treturn lkr.Root()\n\t}\n\n\t// Check if the parent exists:\n\tparent, lerr := lkr.LookupDirectory(dirname)\n\tif lerr != nil && !ie.IsNoSuchFileError(lerr) {\n\t\terr = e.Wrap(lerr, \"dirname lookup failed\")\n\t\treturn\n\t}\n\n\terr = lkr.Atomic(func() (bool, error) {\n\t\t// If it's nil, we might need to create it:\n\t\tif parent == nil {\n\t\t\tif !createParents {\n\t\t\t\treturn false, ie.NoSuchFile(dirname)\n\t\t\t}\n\n\t\t\tparent, err = mkdirParents(lkr, repoPath)\n\t\t\tif err != nil {\n\t\t\t\treturn true, err\n\t\t\t}\n\t\t}\n\n\t\tchild, err := parent.Child(lkr, basename)\n\t\tif err != nil {\n\t\t\treturn true, err\n\t\t}\n\n\t\tif child != nil {\n\t\t\tswitch child.Type() {\n\t\t\tcase n.NodeTypeDirectory:\n\t\t\t\t// Nothing to do really. Return the old child.\n\t\t\t\tdir = child.(*n.Directory)\n\t\t\t\treturn false, nil\n\t\t\tcase n.NodeTypeFile:\n\t\t\t\treturn true, fmt.Errorf(\"`%s` exists and is a file\", repoPath)\n\t\t\tcase n.NodeTypeGhost:\n\t\t\t\t// Remove the ghost and continue with adding:\n\t\t\t\tif err := parent.RemoveChild(lkr, child); err != nil {\n\t\t\t\t\treturn true, err\n\t\t\t\t}\n\t\t\tdefault:\n\t\t\t\treturn true, ie.ErrBadNode\n\t\t\t}\n\t\t}\n\n\t\t// Create it then!\n\t\tdir, err = n.NewEmptyDirectory(lkr, parent, basename, lkr.owner, lkr.NextInode())\n\t\tif err != nil {\n\t\t\treturn true, err\n\t\t}\n\n\t\tif err := lkr.StageNode(dir); err != nil {\n\t\t\treturn true, e.Wrapf(err, \"stage dir\")\n\t\t}\n\n\t\tlog.Debugf(\"mkdir: %s\", dirname)\n\t\treturn false, nil\n\t})\n\n\treturn\n}\n\n// Remove removes a single node from a directory.\n// `nd` is the node that shall be removed and may not be root.\n// The parent directory is returned.\nfunc Remove(lkr *Linker, nd n.ModNode, createGhost, force bool) (parentDir *n.Directory, ghost *n.Ghost, err error) {\n\tif !force && nd.Type() == n.NodeTypeGhost {\n\t\terr = ErrIsGhost\n\t\treturn\n\t}\n\n\tparentDir, err = n.ParentDirectory(lkr, nd)\n\tif err != nil {\n\t\treturn\n\t}\n\n\t// We shouldn't delete the root directory\n\t// (only directory with a parent)\n\tif parentDir == nil {\n\t\terr = fmt.Errorf(\"refusing to delete root\")\n\t\treturn\n\t}\n\n\terr = lkr.Atomic(func() (bool, error) {\n\t\tif err := parentDir.RemoveChild(lkr, nd); err != nil {\n\t\t\treturn true, fmt.Errorf(\"failed to remove child: %v\", err)\n\t\t}\n\n\t\tlkr.MemIndexPurge(nd)\n\n\t\tif err := lkr.StageNode(parentDir); err != nil {\n\t\t\treturn true, err\n\t\t}\n\n\t\tif createGhost {\n\t\t\tnewGhost, err := n.MakeGhost(nd, lkr.NextInode())\n\t\t\tif err != nil {\n\t\t\t\treturn true, err\n\t\t\t}\n\n\t\t\tif err := parentDir.Add(lkr, newGhost); err != nil {\n\t\t\t\treturn true, err\n\t\t\t}\n\n\t\t\tif err := lkr.StageNode(newGhost); err != nil {\n\t\t\t\treturn true, err\n\t\t\t}\n\n\t\t\tghost = newGhost\n\t\t\treturn false, nil\n\t\t}\n\n\t\treturn false, nil\n\t})\n\n\treturn\n}\n\n// prepareParent tries to figure out the correct parent directory when attempting\n// to move `nd` to `dstPath`. It also removes any nodes that are \"in the way\" if possible.\nfunc prepareParent(lkr *Linker, nd n.ModNode, dstPath string) (*n.Directory, error) {\n\t// Check if the destination already exists:\n\tdestNode, err := lkr.LookupModNode(dstPath)\n\tif err != nil && !ie.IsNoSuchFileError(err) {\n\t\treturn nil, err\n\t}\n\n\tif destNode == nil {\n\t\t// No node at this place yet, attempt to look it up.\n\t\treturn lkr.LookupDirectory(path.Dir(dstPath))\n\t}\n\n\tswitch destNode.Type() {\n\tcase n.NodeTypeDirectory:\n\t\t// Move inside of this directory.\n\t\t// Check if there is already a file\n\t\tdestDir, ok := destNode.(*n.Directory)\n\t\tif !ok {\n\t\t\treturn nil, ie.ErrBadNode\n\t\t}\n\n\t\tchild, err := destDir.Child(lkr, nd.Name())\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\t// Oh, something is in there?\n\t\tif child != nil {\n\t\t\tif nd.Type() == n.NodeTypeFile {\n\t\t\t\treturn nil, fmt.Errorf(\n\t\t\t\t\t\"cannot overwrite a directory (%s) with a file (%s)\",\n\t\t\t\t\tdestNode.Path(),\n\t\t\t\t\tchild.Path(),\n\t\t\t\t)\n\t\t\t}\n\n\t\t\tchildDir, ok := child.(*n.Directory)\n\t\t\tif !ok {\n\t\t\t\treturn nil, ie.ErrBadNode\n\t\t\t}\n\n\t\t\tif childDir.Size() > 0 {\n\t\t\t\treturn nil, fmt.Errorf(\n\t\t\t\t\t\"cannot move over: %s; directory is not empty\",\n\t\t\t\t\tchild.Path(),\n\t\t\t\t)\n\t\t\t}\n\n\t\t\t// Okay, there is an empty directory. Let's remove it to\n\t\t\t// replace it with our source node.\n\t\t\tlog.Warningf(\"Remove child dir: %v\", childDir)\n\t\t\tif _, _, err := Remove(lkr, childDir, false, false); err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t}\n\n\t\treturn destDir, nil\n\tcase n.NodeTypeFile:\n\t\tlog.Infof(\"Remove file: %v\", destNode.Path())\n\t\tparentDir, _, err := Remove(lkr, destNode, false, false)\n\t\treturn parentDir, err\n\tcase n.NodeTypeGhost:\n\t\t// It is already a ghost. Overwrite it and do not create a new one.\n\t\tlog.Infof(\"Remove ghost: %v\", destNode.Path())\n\t\tparentDir, _, err := Remove(lkr, destNode, false, true)\n\t\treturn parentDir, err\n\tdefault:\n\t\treturn nil, ie.ErrBadNode\n\t}\n}\n\n// Copy copies the node `nd` to the path at `dstPath`.\nfunc Copy(lkr *Linker, nd n.ModNode, dstPath string) (newNode n.ModNode, err error) {\n\t// Forbid moving a node inside of one of it's subdirectories.\n\tif nd.Path() == dstPath {\n\t\terr = fmt.Errorf(\"source and dest are the same file: %v\", dstPath)\n\t\treturn\n\t}\n\n\tif strings.HasPrefix(path.Dir(dstPath), nd.Path()) {\n\t\terr = fmt.Errorf(\n\t\t\t\"cannot copy `%s` into it's own subdir `%s`\",\n\t\t\tnd.Path(),\n\t\t\tdstPath,\n\t\t)\n\t\treturn\n\t}\n\n\terr = lkr.Atomic(func() (bool, error) {\n\t\tparentDir, err := prepareParent(lkr, nd, dstPath)\n\t\tif err != nil {\n\t\t\treturn true, e.Wrapf(err, \"handle parent\")\n\t\t}\n\n\t\t// We might copy something into a directory.\n\t\t// In this case, dstPath specifies the directory we move into,\n\t\t// not the file we moved to (which we need here)\n\t\tif parentDir.Path() == dstPath {\n\t\t\tdstPath = path.Join(parentDir.Path(), path.Base(nd.Path()))\n\t\t}\n\n\t\t// And add it to the right destination dir:\n\t\tnewNode = nd.Copy(lkr.NextInode())\n\t\tnewNode.SetName(path.Base(dstPath))\n\t\tif err := newNode.SetParent(lkr, parentDir); err != nil {\n\t\t\treturn true, e.Wrapf(err, \"set parent\")\n\t\t}\n\n\t\tif err := newNode.NotifyMove(lkr, parentDir, newNode.Path()); err != nil {\n\t\t\treturn true, e.Wrapf(err, \"notify move\")\n\t\t}\n\n\t\treturn false, lkr.StageNode(newNode)\n\t})\n\n\treturn\n}\n\n// Move moves the node `nd` to the path at `dstPath` and leaves\n// a ghost at the old place.\nfunc Move(lkr *Linker, nd n.ModNode, dstPath string) error {\n\t// Forbid moving a node inside of one of it's subdirectories.\n\tif nd.Type() == n.NodeTypeGhost {\n\t\treturn errors.New(\"cannot move ghosts\")\n\t}\n\n\tif nd.Path() == dstPath {\n\t\treturn fmt.Errorf(\"Source and Dest are the same file: %v\", dstPath)\n\t}\n\n\tif strings.HasPrefix(path.Dir(dstPath), nd.Path()) {\n\t\treturn fmt.Errorf(\n\t\t\t\"Cannot move `%s` into it's own subdir `%s`\",\n\t\t\tnd.Path(),\n\t\t\tdstPath,\n\t\t)\n\t}\n\n\treturn lkr.Atomic(func() (bool, error) {\n\t\tparentDir, err := prepareParent(lkr, nd, dstPath)\n\t\tif err != nil {\n\t\t\treturn true, err\n\t\t}\n\n\t\t// Remove the old node:\n\t\toldPath := nd.Path()\n\t\t_, ghost, err := Remove(lkr, nd, true, true)\n\t\tif err != nil {\n\t\t\treturn true, e.Wrapf(err, \"remove old\")\n\t\t}\n\n\t\tif parentDir.Path() == dstPath {\n\t\t\tdstPath = path.Join(parentDir.Path(), path.Base(oldPath))\n\t\t}\n\n\t\t// The node needs to be told that it's path changed,\n\t\t// since it might need to change it's hash value now.\n\t\tif err := nd.NotifyMove(lkr, parentDir, dstPath); err != nil {\n\t\t\treturn true, e.Wrapf(err, \"notify move\")\n\t\t}\n\n\t\terr = n.Walk(lkr, nd, true, func(child n.Node) error {\n\t\t\treturn e.Wrapf(lkr.StageNode(child), \"stage node\")\n\t\t})\n\n\t\tif err != nil {\n\t\t\treturn true, err\n\t\t}\n\n\t\tif err := lkr.AddMoveMapping(nd.Inode(), ghost.Inode()); err != nil {\n\t\t\treturn true, e.Wrapf(err, \"add move mapping\")\n\t\t}\n\n\t\treturn false, nil\n\t})\n}\n\n// StageFromFileNode is a convinience helper that will call Stage() with all necessary params from `f`.\nfunc StageFromFileNode(lkr *Linker, f *n.File) (*n.File, error) {\n\treturn Stage(\n\t\tlkr,\n\t\tf.Path(),\n\t\tf.ContentHash(),\n\t\tf.BackendHash(),\n\t\tf.Size(),\n\t\tf.CachedSize(),\n\t\tf.Key(),\n\t\tf.ModTime(),\n\t\tf.IsRaw(),\n\t)\n}\n\n// Stage adds a file to brigs DAG this is lesser version since it does not use cachedSize\n// Do not use it if you can, use StageWithFullInfo couple lines below!\n// TODO rename Stage calls everywhere (especially in tests) and then\n// rename Stage -> StageWithoutCacheSize, and StageWithFullInfo -> Stage\nfunc Stage(\n\tlkr *Linker,\n\trepoPath string,\n\tcontentHash,\n\tbackendHash h.Hash,\n\tsize uint64,\n\tcachedSize int64,\n\tkey []byte,\n\tmodTime time.Time,\n\tisRaw bool,\n) (file *n.File, err error) {\n\tnode, lerr := lkr.LookupNode(repoPath)\n\tif lerr != nil && !ie.IsNoSuchFileError(lerr) {\n\t\terr = lerr\n\t\treturn\n\t}\n\n\terr = lkr.Atomic(func() (bool, error) {\n\t\tif node != nil {\n\t\t\tif node.Type() == n.NodeTypeGhost {\n\t\t\t\tghostParent, err := n.ParentDirectory(lkr, node)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn true, err\n\t\t\t\t}\n\n\t\t\t\tif ghostParent == nil {\n\t\t\t\t\treturn true, fmt.Errorf(\n\t\t\t\t\t\t\"bug: %s has no parent. Is root a ghost?\",\n\t\t\t\t\t\tnode.Path(),\n\t\t\t\t\t)\n\t\t\t\t}\n\n\t\t\t\tif err := ghostParent.RemoveChild(lkr, node); err != nil {\n\t\t\t\t\treturn true, err\n\t\t\t\t}\n\n\t\t\t\t// Act like there was no previous node.\n\t\t\t\t// New node will have a different Inode.\n\t\t\t\tfile = nil\n\t\t\t} else {\n\t\t\t\tvar ok bool\n\t\t\t\tfile, ok = node.(*n.File)\n\t\t\t\tif !ok {\n\t\t\t\t\treturn true, ie.ErrBadNode\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\tneedRemove := false\n\t\tif file != nil {\n\t\t\t// We know this file already.\n\t\t\tlog.WithFields(log.Fields{\"file\": repoPath}).Info(\"File exists; modifying.\")\n\t\t\tneedRemove = true\n\n\t\t\tif file.BackendHash().Equal(backendHash) {\n\t\t\t\tlog.Debugf(\"Hash was not modified. Not doing any update.\")\n\t\t\t\treturn false, nil\n\t\t\t}\n\t\t} else {\n\t\t\tparent, err := mkdirParents(lkr, repoPath)\n\t\t\tif err != nil {\n\t\t\t\treturn true, err\n\t\t\t}\n\n\t\t\t// Create a new file at specified path:\n\t\t\tfile = n.NewEmptyFile(parent, path.Base(repoPath), lkr.owner, lkr.NextInode())\n\t\t}\n\n\t\tparentDir, err := n.ParentDirectory(lkr, file)\n\t\tif err != nil {\n\t\t\treturn true, err\n\t\t}\n\n\t\tif parentDir == nil {\n\t\t\treturn true, fmt.Errorf(\"%s has no parent yet (BUG)\", repoPath)\n\t\t}\n\n\t\tif needRemove {\n\t\t\t// Remove the child before changing the hash:\n\t\t\tif err := parentDir.RemoveChild(lkr, file); err != nil {\n\t\t\t\treturn true, err\n\t\t\t}\n\t\t}\n\n\t\tfile.SetSize(size)\n\t\tfile.SetCachedSize(cachedSize)\n\t\tfile.SetModTime(modTime)\n\t\tfile.SetContent(lkr, contentHash)\n\t\tfile.SetBackend(lkr, backendHash)\n\t\tfile.SetKey(key)\n\t\tfile.SetUser(lkr.owner)\n\t\tfile.SetIsRaw(isRaw)\n\n\t\t// Add it again when the hash was changed.\n\t\tlog.Debugf(\"adding %s (%v)\", file.Path(), file.BackendHash())\n\t\tif err := parentDir.Add(lkr, file); err != nil {\n\t\t\treturn true, err\n\t\t}\n\n\t\tif err := lkr.StageNode(file); err != nil {\n\t\t\treturn true, err\n\t\t}\n\n\t\treturn false, nil\n\t})\n\n\treturn\n}\n\n// Log will call `fn` on every commit we currently have, starting\n// with the most current one (CURR, then HEAD, ...).\n// If `fn` will return an error, the iteration is being stopped.\nfunc Log(lkr *Linker, start *n.Commit, fn func(cmt *n.Commit) error) error {\n\tcurr := start\n\tfor curr != nil {\n\t\tif err := fn(curr); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tparent, err := curr.Parent(lkr)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif parent == nil {\n\t\t\tbreak\n\t\t}\n\n\t\tparentCmt, ok := parent.(*n.Commit)\n\t\tif !ok {\n\t\t\treturn ie.ErrBadNode\n\t\t}\n\n\t\tcurr = parentCmt\n\t}\n\n\treturn nil\n}\n" }, { "path": "catfs/core/coreutils_test.go", "content": "package core\n\nimport (\n\t\"path\"\n\t\"sort\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestMkdir(t *testing.T) {\n\tWithDummyLinker(t, func(lkr *Linker) {\n\t\t// Test nested creation without -p like flag:\n\t\tdir, err := Mkdir(lkr, \"/deep/nested\", false)\n\t\tif err == nil || dir != nil {\n\t\t\tt.Fatalf(\"Nested mkdir without -p should have failed: %v\", err)\n\t\t}\n\n\t\tAssertDir(t, lkr, \"/\", true)\n\t\tAssertDir(t, lkr, \"/deep\", false)\n\t\tAssertDir(t, lkr, \"/deep/nested\", false)\n\n\t\t// Test mkdir -p like creating of nested dirs:\n\t\tdir, err = Mkdir(lkr, \"/deep/nested\", true)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"mkdir -p failed: %v\", err)\n\t\t}\n\n\t\tAssertDir(t, lkr, \"/\", true)\n\t\tAssertDir(t, lkr, \"/deep\", true)\n\t\tAssertDir(t, lkr, \"/deep/nested\", true)\n\n\t\t// Attempt to mkdir the same directory once more:\n\t\tdir, err = Mkdir(lkr, \"/deep/nested\", true)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"second mkdir -p failed: %v\", err)\n\t\t}\n\n\t\t// Also without -p, it should just return the respective dir.\n\t\t// (i.e. work like LookupDirectory)\n\t\t// Note: This is a difference to the traditional mkdir.\n\t\tdir, err = Mkdir(lkr, \"/deep/nested\", false)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"second mkdir without -p failed: %v\", err)\n\t\t}\n\n\t\t// See if an attempt at creating the root failed,\n\t\t// should not and just work like lkr.LookupDirectory(\"/\")\n\t\tdir, err = Mkdir(lkr, \"/\", false)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"mkdir root failed (without -p): %v\", err)\n\t\t}\n\n\t\troot, err := lkr.Root()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to retrieve root: %v\", err)\n\t\t}\n\n\t\tif !dir.TreeHash().Equal(root.TreeHash()) {\n\t\t\tt.Fatal(\"Root and mkdir('/') differ!\")\n\t\t}\n\n\t\t// Try to mkdir over a regular file:\n\t\tMustTouch(t, lkr, \"/cat.png\", 1)\n\n\t\t// This should fail, since we cannot create it.\n\t\tdir, err = Mkdir(lkr, \"/cat.png\", false)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"Creating directory on file should have failed!\")\n\t\t}\n\n\t\t// Same even for -p\n\t\tdir, err = Mkdir(lkr, \"/cat.png\", true)\n\t\tif err == nil {\n\t\t\tt.Fatal(\"Creating directory on file should have failed!\")\n\t\t}\n\t})\n}\n\nfunc TestRemove(t *testing.T) {\n\tWithDummyLinker(t, func(lkr *Linker) {\n\t\tdir, err := Mkdir(lkr, \"/some/nested/directory\", true)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to mkdir a nested directory: %v\", err)\n\t\t}\n\n\t\tAssertDir(t, lkr, \"/some/nested/directory\", true)\n\n\t\tpath := \"/some/nested/directory/cat.png\"\n\t\tMustTouch(t, lkr, path, 1)\n\n\t\t// Check file removal with ghost creation:\n\t\tfile, err := lkr.LookupFile(path)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to lookup nested file: %v\", err)\n\t\t}\n\n\t\t// Fill in a dummy file hash, so we get a ghost instance\n\t\tparentDir, _, err := Remove(lkr, file, true, false)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Remove failed: %v\", err)\n\t\t}\n\n\t\tif !parentDir.TreeHash().Equal(dir.TreeHash()) {\n\t\t\tt.Fatalf(\"Hash differs on %s and %s\", dir.Path(), parentDir.TreeHash())\n\t\t}\n\n\t\t// Check that a ghost was created for the removed file:\n\n\t\tghost, err := lkr.LookupGhost(path)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Looking up ghost failed: %v\", err)\n\t\t}\n\n\t\toldFile, err := ghost.OldFile()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to retrieve old file from ghost: %v\", err)\n\t\t}\n\n\t\tif !oldFile.TreeHash().Equal(file.TreeHash()) {\n\t\t\tt.Fatal(\"Old file and original file hashes differ!\")\n\t\t}\n\n\t\t// Check directory removal:\n\t\tnestedDir, err := lkr.LookupDirectory(\"/some/nested\")\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Lookup on /some/nested failed: %v\", err)\n\t\t}\n\n\t\tnestedParentDir, err := nestedDir.Parent(lkr)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Getting parent of /some/nested failed: %v\", err)\n\t\t}\n\n\t\t// Just fill in a dummy moved to ref, to get a ghost.\n\t\tparentDir, ghost, err = Remove(lkr, nestedDir, true, false)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Directory removal failed: %v\", err)\n\t\t}\n\n\t\tif ghost == nil || ghost.Type() != n.NodeTypeGhost {\n\t\t\tt.Fatalf(\"Ghost node does not look like a ghost: %v\", ghost)\n\t\t}\n\n\t\tif !parentDir.TreeHash().Equal(nestedParentDir.TreeHash()) {\n\t\t\tt.Fatalf(\"Hash differs on %s and %s\", nestedParentDir.Path(), parentDir.TreeHash())\n\t\t}\n\t})\n}\n\nfunc TestRemoveGhost(t *testing.T) {\n\tWithDummyLinker(t, func(lkr *Linker) {\n\t\tfile := MustTouch(t, lkr, \"/x\", 1)\n\t\tpar, err := n.ParentDirectory(lkr, file)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to get get parent directory of /x: %v\", err)\n\t\t}\n\n\t\tif err := par.RemoveChild(lkr, file); err != nil {\n\t\t\tt.Fatalf(\"Removing child /x failed: %v\", err)\n\t\t}\n\n\t\tghost, err := n.MakeGhost(file, 42)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to summon ghost: %v\", err)\n\t\t}\n\n\t\tif err := par.Add(lkr, ghost); err != nil {\n\t\t\tt.Fatalf(\"Re-adding ghost failed: %v\", err)\n\t\t}\n\n\t\tif err := lkr.StageNode(ghost); err != nil {\n\t\t\tt.Fatalf(\"Staging ghost failed: %v\", err)\n\t\t}\n\n\t\t// Try to remove a ghost:\n\t\tif _, _, err := Remove(lkr, ghost, true, false); err != ErrIsGhost {\n\t\t\tt.Fatalf(\"Removing ghost failed other than expected: %v\", err)\n\t\t}\n\t})\n}\n\nfunc TestRemoveExistingGhost(t *testing.T) {\n\tWithDummyLinker(t, func(lkr *Linker) {\n\t\tnd := MustTouch(t, lkr, \"/x\", 1)\n\t\t_, ghost, err := Remove(lkr, nd, true, true)\n\t\trequire.Nil(t, err)\n\n\t\t_, _, err = Remove(lkr, ghost, false, true)\n\t\trequire.Nil(t, err)\n\n\t\t_, _, err = Remove(lkr, ghost, true, true)\n\t\trequire.NotNil(t, err)\n\t})\n}\n\nfunc moveValidCheck(t *testing.T, lkr *Linker, srcPath, dstPath string) {\n\tnd, err := lkr.LookupNode(srcPath)\n\n\tif err == nil {\n\t\tif nd.Type() != n.NodeTypeGhost {\n\t\t\tt.Fatalf(\"Source node still exists! (%v): %v\", srcPath, nd.Type())\n\t\t}\n\t} else if !ie.IsNoSuchFileError(err) {\n\t\tt.Fatalf(\"Looking up source node failed: %v\", err)\n\t}\n\n\tlkDestNode, err := lkr.LookupNode(dstPath)\n\tif err != nil {\n\t\tt.Fatalf(\"Looking up dest path failed: %v\", err)\n\t}\n\n\tif lkDestNode.Path() != dstPath {\n\t\tt.Fatalf(\"Dest nod and dest path differ: %v <-> %v\", lkDestNode.Path(), dstPath)\n\t}\n}\n\nfunc moveInvalidCheck(t *testing.T, lkr *Linker, srcPath, dstPath string) {\n\tnode, err := lkr.LookupNode(srcPath)\n\tif err != nil {\n\t\tt.Fatalf(\"Source node vanished during errorneous move: %v\", err)\n\t}\n\n\tif node.Type() == n.NodeTypeGhost {\n\t\tt.Fatalf(\"Source node was converted to a ghost: %v\", node.Path())\n\t}\n}\n\nvar moveAndCopyTestCases = []struct {\n\tname string\n\tisErrorCase bool\n\tsetup func(t *testing.T, lkr *Linker) (n.ModNode, string)\n}{\n\t{\n\t\tname: \"basic\",\n\t\tisErrorCase: false,\n\t\tsetup: func(t *testing.T, lkr *Linker) (n.ModNode, string) {\n\t\t\tMustMkdir(t, lkr, \"/a/b/c\")\n\t\t\treturn MustTouch(t, lkr, \"/a/b/c/x\", 1), \"/a/b/y\"\n\t\t},\n\t}, {\n\t\tname: \"basic-directory\",\n\t\tisErrorCase: false,\n\t\tsetup: func(t *testing.T, lkr *Linker) (n.ModNode, string) {\n\t\t\treturn MustMkdir(t, lkr, \"/a/b/short\"), \"/a/b/looooong\"\n\t\t},\n\t}, {\n\t\tname: \"basic-same-level\",\n\t\tisErrorCase: false,\n\t\tsetup: func(t *testing.T, lkr *Linker) (n.ModNode, string) {\n\t\t\treturn MustTouch(t, lkr, \"/a\", 1), \"/b\"\n\t\t},\n\t}, {\n\t\tname: \"basic-root-to-sub\",\n\t\tisErrorCase: false,\n\t\tsetup: func(t *testing.T, lkr *Linker) (n.ModNode, string) {\n\t\t\tMustTouch(t, lkr, \"/README.md\", 1)\n\t\t\tMustMkdir(t, lkr, \"/sub\")\n\t\t\treturn MustTouch(t, lkr, \"/x\", 1), \"/sub\"\n\t\t},\n\t}, {\n\t\tname: \"into-directory\",\n\t\tisErrorCase: false,\n\t\tsetup: func(t *testing.T, lkr *Linker) (n.ModNode, string) {\n\t\t\tMustMkdir(t, lkr, \"/a/b/c\")\n\t\t\tMustMkdir(t, lkr, \"/a/b/d\")\n\t\t\treturn MustTouch(t, lkr, \"/a/b/c/x\", 1), \"/a/b/d\"\n\t\t},\n\t}, {\n\t\tname: \"into-nonempty-directory\",\n\t\tisErrorCase: false,\n\t\tsetup: func(t *testing.T, lkr *Linker) (n.ModNode, string) {\n\t\t\tMustMkdir(t, lkr, \"/a/b/c\")\n\t\t\tMustMkdir(t, lkr, \"/a/b/d\")\n\t\t\tMustTouch(t, lkr, \"/a/b/d/y\", 1)\n\t\t\treturn MustTouch(t, lkr, \"/a/b/c/x\", 1), \"/a/b/d\"\n\t\t},\n\t}, {\n\t\tname: \"error-to-directory-contains-file\",\n\t\tisErrorCase: true,\n\t\tsetup: func(t *testing.T, lkr *Linker) (n.ModNode, string) {\n\t\t\tMustMkdir(t, lkr, \"/src\")\n\t\t\tMustMkdir(t, lkr, \"/dst\")\n\t\t\tMustTouch(t, lkr, \"/dst/x\", 1)\n\t\t\treturn MustTouch(t, lkr, \"/src/x\", 1), \"/dst\"\n\t\t},\n\t}, {\n\t\tname: \"error-file-over-existing\",\n\t\tisErrorCase: false,\n\t\tsetup: func(t *testing.T, lkr *Linker) (n.ModNode, string) {\n\t\t\tMustMkdir(t, lkr, \"/src\")\n\t\t\tMustMkdir(t, lkr, \"/dst\")\n\t\t\tMustTouch(t, lkr, \"/dst/x\", 1)\n\t\t\treturn MustTouch(t, lkr, \"/src/x\", 1), \"/dst/x\"\n\t\t},\n\t}, {\n\t\tname: \"error-file-over-ghost\",\n\t\tisErrorCase: false,\n\t\tsetup: func(t *testing.T, lkr *Linker) (n.ModNode, string) {\n\t\t\tMustMkdir(t, lkr, \"/src\")\n\t\t\tMustMkdir(t, lkr, \"/dst\")\n\t\t\tdestFile := MustTouch(t, lkr, \"/dst/x\", 1)\n\t\t\tMustRemove(t, lkr, destFile)\n\t\t\treturn MustTouch(t, lkr, \"/src/x\", 1), \"/dst/x\"\n\t\t},\n\t}, {\n\t\tname: \"error-src-equal-dst\",\n\t\tisErrorCase: true,\n\t\tsetup: func(t *testing.T, lkr *Linker) (n.ModNode, string) {\n\t\t\treturn MustTouch(t, lkr, \"/x\", 1), \"/x\"\n\t\t},\n\t}, {\n\t\tname: \"error-into-own-subdir\",\n\t\tisErrorCase: true,\n\t\tsetup: func(t *testing.T, lkr *Linker) (n.ModNode, string) {\n\t\t\t// We should not be able to move \"/dir\" into itself.\n\t\t\tdir := MustMkdir(t, lkr, \"/dir\")\n\t\t\tMustTouch(t, lkr, \"/dir/x\", 1)\n\t\t\treturn dir, \"/dir/own\"\n\t\t},\n\t},\n}\n\nfunc TestMoveSingle(t *testing.T) {\n\t// Cases to cover for move():\n\t// 1. Dest exists:\n\t// 1.1. Is a directory.\n\t// 1.1.1 E This directory contains basename(src) and it is a file.\n\t// 1.1.2 E This directory contains basename(src) and it is a non-empty dir.\n\t// 1.1.3 V This directory contains basename(src) and it is a empty dir.\n\t// 2. Dest does not exist.\n\t// 2.1 V dirname(dest) exists and is a directory.\n\t// 2.2 E dirname(dest) does not exists.\n\t// 2.2 E dirname(dest) exists and is not a directory.\n\t// 3. E Overlap of src and dest paths (src in dest)\n\n\t// Checks for valid cases (V):\n\t// 1) src is gone.\n\t// 2) dest is the same node as before.\n\t// 3) dest has the correct path.\n\n\t// Checks for invalid cases (E):\n\t// 1) src is not gone.\n\n\tfor _, tc := range moveAndCopyTestCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tWithDummyLinker(t, func(lkr *Linker) {\n\t\t\t\t// Setup src and dest dir with a file in it named like src.\n\t\t\t\tsrcNd, dstPath := tc.setup(t, lkr)\n\t\t\t\tsrcPath := srcNd.Path()\n\n\t\t\t\tif err := Move(lkr, srcNd, dstPath); err != nil {\n\t\t\t\t\tif tc.isErrorCase {\n\t\t\t\t\t\tmoveInvalidCheck(t, lkr, srcPath, dstPath)\n\t\t\t\t\t} else {\n\t\t\t\t\t\tt.Fatalf(\"Move failed unexpectly: %v\", err)\n\t\t\t\t\t}\n\t\t\t\t} else {\n\t\t\t\t\tmoveValidCheck(t, lkr, srcPath, dstPath)\n\t\t\t\t}\n\t\t\t})\n\t\t})\n\t}\n}\n\nfunc TestMoveDirectoryWithChild(t *testing.T) {\n\tWithDummyLinker(t, func(lkr *Linker) {\n\t\tMustMkdir(t, lkr, \"/src\")\n\t\toldFile := MustTouch(t, lkr, \"/src/x\", 1)\n\t\toldFile = oldFile.Copy(oldFile.Inode()).(*n.File)\n\n\t\tMustCommit(t, lkr, \"before move\")\n\n\t\tdir, err := lkr.LookupDirectory(\"/src\")\n\t\trequire.Nil(t, err)\n\n\t\tMustMove(t, lkr, dir, \"/dst\")\n\t\tMustCommit(t, lkr, \"after move\")\n\n\t\tfile, err := lkr.LookupFile(\"/dst/x\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, h.TestDummy(t, 1), file.BackendHash())\n\n\t\t_, err = lkr.LookupGhost(\"/src\")\n\t\trequire.Nil(t, err)\n\n\t\t// This will resolve to the old file:\n\t\t_, err = lkr.LookupFile(\"/src/x\")\n\t\trequire.NotNil(t, err)\n\t})\n}\n\nfunc TestMoveDirectory(t *testing.T) {\n\tWithDummyLinker(t, func(lkr *Linker) {\n\t\tsrcDir := MustMkdir(t, lkr, \"/src\")\n\t\tMustMkdir(t, lkr, \"/src/sub\")\n\t\tMustTouch(t, lkr, \"/src/sub/x\", 23)\n\t\tMustTouch(t, lkr, \"/src/y\", 23)\n\n\t\tdstDir := MustMove(t, lkr, srcDir, \"/dst\")\n\n\t\texpect := []string{\n\t\t\t\"/dst/sub/x\",\n\t\t\t\"/dst/sub\",\n\t\t\t\"/dst/y\",\n\t\t\t\"/dst\",\n\t\t}\n\n\t\tgot := []string{}\n\t\trequire.Nil(t, n.Walk(lkr, dstDir, true, func(child n.Node) error {\n\t\t\tgot = append(got, child.Path())\n\t\t\treturn nil\n\t\t}))\n\n\t\tsort.Strings(expect)\n\t\tsort.Strings(got)\n\n\t\trequire.Equal(t, len(expect), len(got))\n\t\tfor idx := range expect {\n\t\t\tif got[idx] != expect[idx] {\n\t\t\t\tt.Errorf(\n\t\t\t\t\t\"Moved node child `%s` does not match `%s`\",\n\t\t\t\t\tgot[idx],\n\t\t\t\t\texpect[idx],\n\t\t\t\t)\n\t\t\t}\n\t\t}\n\t})\n}\n\nfunc TestMoveDirectoryWithGhosts(t *testing.T) {\n\tWithDummyLinker(t, func(lkr *Linker) {\n\t\tsrcDir := MustMkdir(t, lkr, \"/src\")\n\t\tMustMkdir(t, lkr, \"/src/sub\")\n\t\txFile := MustTouch(t, lkr, \"/src/sub/x\", 23)\n\t\tMustTouch(t, lkr, \"/src/y\", 23)\n\t\tMustMove(t, lkr, xFile, \"/src/z\")\n\n\t\tdstDir := MustMove(t, lkr, srcDir, \"/dst\")\n\n\t\texpect := []string{\n\t\t\t\"/dst\",\n\t\t\t\"/dst/sub\",\n\t\t\t\"/dst/sub/x\",\n\t\t\t\"/dst/y\",\n\t\t\t\"/dst/z\",\n\t\t}\n\n\t\t// Be evil and clear the mem cache in order to check if all changes\n\t\t// were checked into the staging area.\n\t\tlkr.MemIndexClear()\n\n\t\tgot := []string{}\n\t\trequire.Nil(t, n.Walk(lkr, dstDir, true, func(child n.Node) error {\n\t\t\tgot = append(got, child.Path())\n\t\t\treturn nil\n\t\t}))\n\n\t\t// Check if the moved directory contains the right paths:\n\t\tsort.Strings(got)\n\t\tfor idx, expectPath := range expect {\n\t\t\tif expectPath != got[idx] {\n\t\t\t\tt.Fatalf(\"%d: %s != %s\", idx, expectPath, got[idx])\n\t\t\t}\n\t\t}\n\n\t\tghost, err := lkr.LookupNode(got[2])\n\t\trequire.Nil(t, err)\n\n\t\tstatus, err := lkr.Status()\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, \"/src/sub/x\", ghost.(*n.Ghost).OldNode().Path())\n\n\t\ttwin, _, err := lkr.MoveMapping(status, ghost)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, \"/dst/z\", twin.Path())\n\t})\n}\n\nfunc TestStage(t *testing.T) {\n\tWithDummyLinker(t, func(lkr *Linker) {\n\t\t// Initial stage of the file:\n\t\tkey := make([]byte, 32)\n\n\t\tcontentHash1 := h.TestDummy(t, 1)\n\t\tbackendHash1 := h.TestDummy(t, 1)\n\t\tfile, err := Stage(\n\t\t\tlkr,\n\t\t\t\"/photos/moose.png\",\n\t\t\tcontentHash1,\n\t\t\tbackendHash1,\n\t\t\t2,\n\t\t\t-1,\n\t\t\tkey,\n\t\t\ttime.Now(),\n\t\t\tfalse,\n\t\t)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Adding of /photos/moose.png failed: %v\", err)\n\t\t}\n\n\t\tcontentHash2 := h.TestDummy(t, 2)\n\t\tbackendHash2 := h.TestDummy(t, 2)\n\t\tfile, err = Stage(\n\t\t\tlkr,\n\t\t\t\"/photos/moose.png\",\n\t\t\tcontentHash2,\n\t\t\tbackendHash2,\n\t\t\t3,\n\t\t\t-1,\n\t\t\tkey,\n\t\t\ttime.Now(),\n\t\t\tfalse,\n\t\t)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Adding of /photos/moose.png failed: %v\", err)\n\t\t}\n\n\t\tif !file.BackendHash().Equal(h.TestDummy(t, 2)) {\n\t\t\tt.Fatalf(\n\t\t\t\t\"File content after update is not what's advertised: %v\",\n\t\t\t\tfile.TreeHash(),\n\t\t\t)\n\t\t}\n\t})\n}\n\nfunc TestStageDirOverGhost(t *testing.T) {\n\tWithDummyLinker(t, func(lkr *Linker) {\n\t\tempty := MustMkdir(t, lkr, \"/empty\")\n\t\tMustMove(t, lkr, empty, \"/moved_empty\")\n\t\tMustMkdir(t, lkr, \"/empty\")\n\t\tdir, err := lkr.LookupDirectory(\"/empty\")\n\t\trequire.Nil(t, err)\n\n\t\trequire.Equal(t, dir.Path(), \"/empty\")\n\t\tif dir.Type() != n.NodeTypeDirectory {\n\t\t\tt.Fatalf(\"/empty is not a directory\")\n\t\t}\n\t})\n}\n\nfunc TestCopy(t *testing.T) {\n\tfor _, tc := range moveAndCopyTestCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\n\t\t\tWithDummyLinker(t, func(lkr *Linker) {\n\t\t\t\t// Setup src and dest dir with a file in it named like src.\n\t\t\t\tsrcNd, dstPath := tc.setup(t, lkr)\n\t\t\t\tsrcPath := srcNd.Path()\n\n\t\t\t\tnewNd, err := Copy(lkr, srcNd, dstPath)\n\t\t\t\tif newNd != nil {\n\t\t\t\t\tif !strings.HasPrefix(newNd.Path(), dstPath) {\n\t\t\t\t\t\tt.Fatalf(\n\t\t\t\t\t\t\t\"Node was copied to wrong path: %v (want %v)\",\n\t\t\t\t\t\t\tnewNd.Path(),\n\t\t\t\t\t\t\tdstPath,\n\t\t\t\t\t\t)\n\t\t\t\t\t}\n\n\t\t\t\t\t// Make sure the new copy is reachable from parent:\n\t\t\t\t\tpar, err := lkr.LookupDirectory(path.Dir(newNd.Path()))\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\tt.Fatalf(\"Failed to lookup parent: %v\", err)\n\t\t\t\t\t}\n\n\t\t\t\t\tnewChildNd, err := par.Child(lkr, newNd.Name())\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\tt.Fatalf(\"Failed to get base path: %v\", err)\n\t\t\t\t\t}\n\n\t\t\t\t\tnewNd = newChildNd.(n.ModNode)\n\t\t\t\t}\n\n\t\t\t\tif oldNd, lookErr := lkr.LookupNode(srcPath); oldNd == nil || lookErr != nil {\n\t\t\t\t\tt.Fatalf(\"source node does not exist or is not accesible: %v %v\", err, tc.isErrorCase)\n\t\t\t\t}\n\n\t\t\t\tif err != nil {\n\t\t\t\t\tif tc.isErrorCase {\n\t\t\t\t\t\tnode, err := lkr.LookupNode(srcPath)\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\tt.Fatalf(\"Source node vanished during errorneous copy: %v\", err)\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\tif node.Type() == n.NodeTypeGhost {\n\t\t\t\t\t\t\tt.Fatalf(\"Source node was converted to a ghost: %v\", node.Path())\n\t\t\t\t\t\t}\n\t\t\t\t\t} else {\n\t\t\t\t\t\tt.Fatalf(\"Copy failed unexpectly: %v\", err)\n\t\t\t\t\t}\n\n\t\t\t\t\t// No need to test more.\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif tc.isErrorCase {\n\t\t\t\t\tt.Fatalf(\"test should have failed\")\n\t\t\t\t}\n\n\t\t\t\tif newNd == nil {\n\t\t\t\t\tt.Fatalf(\"Dest node does not exist after copy: %v\", err)\n\t\t\t\t}\n\n\t\t\t\tif !newNd.BackendHash().Equal(srcNd.BackendHash()) {\n\t\t\t\t\tt.Logf(\"Content of src and dst differ after copy\")\n\t\t\t\t\tt.Logf(\"WANT: %v\", srcNd.BackendHash())\n\t\t\t\t\tt.Logf(\"GOT : %v\", newNd.BackendHash())\n\t\t\t\t\tt.Fatalf(\"Check Copy()\")\n\t\t\t\t}\n\n\t\t\t\tif newNd.Inode() < srcNd.Inode() {\n\t\t\t\t\tt.Fatalf(\"New inode has <= inode of src\")\n\t\t\t\t}\n\n\t\t\t\t// Sanity check: do not rely on Copy() to return a valid, staged node.\n\t\t\t\t// Check if we can look it up after the Copy too.\n\t\t\t\tnd, err := lkr.LookupNode(newNd.Path())\n\t\t\t\trequire.Nil(t, err)\n\t\t\t\trequire.NotNil(t, nd)\n\t\t\t})\n\t\t})\n\t}\n}\n" }, { "path": "catfs/core/gc.go", "content": "package core\n\nimport (\n\t\"github.com/sahib/brig/catfs/db\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// GarbageCollector implements a small mark & sweep garbage collector.\n// It exists more for the sake of fault tolerance than it being an\n// essential part of brig. This is different from the ipfs garbage collector.\ntype GarbageCollector struct {\n\tlkr *Linker\n\tkv db.Database\n\tnotifier func(nd n.Node) bool\n\tmarkMap map[string]struct{}\n}\n\n// NewGarbageCollector will return a new GC, operating on `lkr` and `kv`.\n// It will call `kc` on every collected node.\nfunc NewGarbageCollector(lkr *Linker, kv db.Database, kc func(nd n.Node) bool) *GarbageCollector {\n\treturn &GarbageCollector{\n\t\tlkr: lkr,\n\t\tkv: kv,\n\t\tnotifier: kc,\n\t}\n}\n\nfunc (gc *GarbageCollector) markMoveMap(key []string) error {\n\tkeys, err := gc.kv.Keys(key...)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor _, key := range keys {\n\t\tdata, err := gc.kv.Get(key...)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tnode, _, err := gc.lkr.parseMoveMappingLine(string(data))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif node != nil {\n\t\t\tgc.markMap[node.TreeHash().B58String()] = struct{}{}\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (gc *GarbageCollector) mark(cmt *n.Commit, recursive bool) error {\n\tif cmt == nil {\n\t\treturn nil\n\t}\n\n\troot, err := gc.lkr.DirectoryByHash(cmt.Root())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tgc.markMap[cmt.TreeHash().B58String()] = struct{}{}\n\terr = n.Walk(gc.lkr, root, true, func(child n.Node) error {\n\t\tgc.markMap[child.TreeHash().B58String()] = struct{}{}\n\t\treturn nil\n\t})\n\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tparent, err := cmt.Parent(gc.lkr)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif recursive && parent != nil {\n\t\tparentCmt, ok := parent.(*n.Commit)\n\t\tif !ok {\n\t\t\treturn ie.ErrBadNode\n\t\t}\n\n\t\treturn gc.mark(parentCmt, recursive)\n\t}\n\n\treturn nil\n}\n\nfunc (gc *GarbageCollector) sweep(prefix []string) (int, error) {\n\tremoved := 0\n\n\treturn removed, gc.lkr.AtomicWithBatch(func(batch db.Batch) (bool, error) {\n\t\tkeys, err := gc.kv.Keys(prefix...)\n\t\tif err != nil {\n\t\t\treturn hintRollback(err)\n\t\t}\n\n\t\tfor _, key := range keys {\n\t\t\tb58Hash := key[len(key)-1]\n\t\t\tif _, ok := gc.markMap[b58Hash]; ok {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\thash, err := h.FromB58String(b58Hash)\n\t\t\tif err != nil {\n\t\t\t\treturn hintRollback(err)\n\t\t\t}\n\n\t\t\tnode, err := gc.lkr.NodeByHash(hash)\n\t\t\tif err != nil {\n\t\t\t\treturn hintRollback(err)\n\t\t\t}\n\n\t\t\tif node == nil {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\t// Allow the gc caller to check if he really\n\t\t\t// wants to delete this node.\n\t\t\tif gc.notifier != nil && !gc.notifier(node) {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\t// Actually get rid of the node:\n\t\t\tgc.lkr.MemIndexPurge(node)\n\n\t\t\tbatch.Erase(key...)\n\t\t\tremoved++\n\t\t}\n\n\t\treturn false, nil\n\t})\n}\n\nfunc (gc *GarbageCollector) findAllMoveLocations(head *n.Commit) ([][]string, error) {\n\tlocations := [][]string{\n\t\t{\"stage\", \"moves\"},\n\t}\n\n\tfor {\n\t\tparent, err := head.Parent(gc.lkr)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif parent == nil {\n\t\t\tbreak\n\t\t}\n\n\t\tparentCmt, ok := parent.(*n.Commit)\n\t\tif !ok {\n\t\t\treturn nil, ie.ErrBadNode\n\t\t}\n\n\t\thead = parentCmt\n\t\tlocation := []string{\"moves\", head.TreeHash().B58String()}\n\t\tlocations = append(locations, location)\n\t}\n\n\treturn locations, nil\n}\n\n// Run will trigger a GC run. If `allObjects` is false,\n// only the staging commit will be checked. Otherwise\n// all objects in the key value store.\nfunc (gc *GarbageCollector) Run(allObjects bool) error {\n\tgc.markMap = make(map[string]struct{})\n\thead, err := gc.lkr.Status()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif err := gc.mark(head, allObjects); err != nil {\n\t\treturn err\n\t}\n\n\t// Staging might contain moved files that are not reachable anymore,\n\t// but still are referenced by the move mapping.\n\t// Keep them for now, they will die most likely on MakeCommit()\n\tmoveMapLocations := [][]string{\n\t\t{\"stage\", \"moves\"},\n\t}\n\n\tif allObjects {\n\t\tmoveMapLocations, err = gc.findAllMoveLocations(head)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tfor _, location := range moveMapLocations {\n\t\tif err := gc.markMoveMap(location); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tremoved, err := gc.sweep([]string{\"stage\", \"objects\"})\n\tif err != nil {\n\t\tlog.Debugf(\"removed %d unreachable staging objects.\", removed)\n\t}\n\n\tif allObjects {\n\t\tremoved, err = gc.sweep([]string{\"objects\"})\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif removed > 0 {\n\t\t\tlog.Warningf(\"removed %d unreachable permanent objects.\", removed)\n\t\t\tlog.Warningf(\"this might indiciate a bug in catfs somewhere.\")\n\t\t}\n\t}\n\n\treturn nil\n}\n" }, { "path": "catfs/core/gc_test.go", "content": "package core\n\nimport (\n\t\"testing\"\n\n\t\"github.com/sahib/brig/catfs/db\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc assertNodeExists(t *testing.T, kv db.Database, nd n.Node) {\n\tif _, err := kv.Get(\"stage\", \"objects\", nd.TreeHash().B58String()); err != nil {\n\t\tt.Fatalf(\"Stage object %v does not exist: %v\", nd, err)\n\t}\n}\n\nfunc TestGC(t *testing.T) {\n\tmdb := db.NewMemoryDatabase()\n\tlkr := NewLinker(mdb)\n\n\tkillExpected := make(map[string]bool)\n\tkillActual := make(map[string]bool)\n\n\tgc := NewGarbageCollector(lkr, mdb, func(nd n.Node) bool {\n\t\tkillActual[nd.TreeHash().B58String()] = true\n\t\treturn true\n\t})\n\n\troot, err := lkr.Root()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to retrieve the root: %v\", root)\n\t}\n\n\tkillExpected[root.TreeHash().B58String()] = true\n\n\tsub1, err := n.NewEmptyDirectory(lkr, root, \"a\", \"u\", 3)\n\tif err != nil {\n\t\tt.Fatalf(\"Creating sub2 failed: %v\", err)\n\t}\n\n\tif err := lkr.StageNode(sub1); err != nil {\n\t\tt.Fatalf(\"Staging root failed: %v\", err)\n\t}\n\n\tkillExpected[root.TreeHash().B58String()] = true\n\tkillExpected[sub1.TreeHash().B58String()] = true\n\n\tsub2, err := n.NewEmptyDirectory(lkr, sub1, \"b\", \"u\", 4)\n\tif err != nil {\n\t\tt.Fatalf(\"Creating sub2 failed: %v\", err)\n\t}\n\n\tif err := lkr.StageNode(sub2); err != nil {\n\t\tt.Fatalf(\"Staging root failed: %v\", err)\n\t}\n\n\troot, err = lkr.Root()\n\trequire.Nil(t, err)\n\n\tif err := gc.Run(true); err != nil {\n\t\tt.Fatalf(\"gc run failed: %v\", err)\n\t}\n\n\tif len(killExpected) != len(killActual) {\n\t\tt.Fatalf(\n\t\t\t\"GC killed %d nodes, but should have killed %d\",\n\t\t\tlen(killActual),\n\t\t\tlen(killExpected),\n\t\t)\n\t}\n\n\tfor killedHash := range killActual {\n\t\tif _, ok := killExpected[killedHash]; !ok {\n\t\t\tt.Fatalf(\"%s was killed, but should not!\", killedHash)\n\t\t}\n\n\t\tif _, err := mdb.Get(\"stage\", \"objects\", killedHash); err != db.ErrNoSuchKey {\n\t\t\tt.Fatalf(\"GC did not wipe key from db: %v\", killedHash)\n\t\t}\n\t}\n\n\t// Double check that the gc did not delete other stuff from the db:\n\tassertNodeExists(t, mdb, root)\n\tassertNodeExists(t, mdb, sub1)\n\tassertNodeExists(t, mdb, sub2)\n\n\tgc = NewGarbageCollector(lkr, mdb, func(nd n.Node) bool {\n\t\tt.Fatalf(\"Second gc run found something, first didn't\")\n\t\treturn true\n\t})\n\n\tif err := gc.Run(true); err != nil {\n\t\tt.Fatalf(\"Second gc run failed: %v\", err)\n\t}\n\n\tif err := lkr.MakeCommit(n.AuthorOfStage, \"some message\"); err != nil {\n\t\tt.Fatalf(\"MakeCommit() failed: %v\", err)\n\t}\n\n\tgc = NewGarbageCollector(lkr, mdb, func(nd n.Node) bool {\n\t\tt.Fatalf(\"Third gc run found something, first didn't\")\n\t\treturn true\n\t})\n\n\tif err := gc.Run(true); err != nil {\n\t\tt.Fatalf(\"Third gc run failed: %v\", err)\n\t}\n}\n" }, { "path": "catfs/core/linker.go", "content": "package core\n\n// Layout of the key/value store:\n//\n// objects/ => NODE_METADATA\n// tree/ => NODE_HASH\n// index/ => COMMIT_HASH\n// inode/ => NODE_HASH\n// moves/ => MOVE_INFO\n// moves/overlay/ => MOVE_INFO\n//\n// stage/objects/ => NODE_METADATA\n// stage/tree/ => NODE_HASH\n// stage/STATUS => COMMIT_METADATA\n// stage/moves/ => MOVE_INFO\n// stage/moves/overlay/ => MOVE_INFO\n//\n// stats/max-inode => UINT64\n// refs/ => NODE_HASH\n//\n// Defined by caller:\n//\n// metadata/ => BYTES (Caller defined data)\n//\n// NODE is either a Commit, a Directory or a File.\n// FULL_NODE_PATH may contain slashes and in case of directories,\n// it will contain a trailing slash.\n//\n// The following refs are defined by the system:\n// HEAD -> Points to the latest finished commit, or nil.\n// CURR -> Points to the staging commit.\n//\n// In git terminology, this file implements the following commands:\n//\n// - git add: StageNode(): Create and Update Nodes.\n// - git status: Status()\n// - git commit: MakeCommit()\n//\n// All write operations are written in one batch or are rolled back\n// on errors.\n\nimport (\n\t\"encoding/binary\"\n\t\"fmt\"\n\t\"path\"\n\t\"runtime/debug\"\n\t\"strconv\"\n\t\"strings\"\n\t\"time\"\n\n\te \"github.com/pkg/errors\"\n\t\"github.com/sahib/brig/catfs/db\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\t\"github.com/sahib/brig/util/trie\"\n\tlog \"github.com/sirupsen/logrus\"\n\tcapnp \"zombiezen.com/go/capnproto2\"\n)\n\n// Linker implements the basic logic of brig's data model\n// It uses an underlying key/value database to\n// storea a Merkle-DAG with versioned metadata,\n// similar to what git does internally.\ntype Linker struct {\n\tkv db.Database\n\n\t// root of the filesystem\n\troot *n.Directory\n\n\t// Path lookup trie\n\tptrie *trie.Node\n\n\t// B58Hash to node\n\tindex map[string]n.Node\n\n\t// UID to node\n\tinodeIndex map[uint64]n.Node\n\n\t// Cache for the linker owner.\n\towner string\n}\n\n// NewLinker returns a new lkr, ready to use. It assumes the key value store\n// is working and does no check on this.\nfunc NewLinker(kv db.Database) *Linker {\n\tlkr := &Linker{kv: kv}\n\tlkr.MemIndexClear()\n\treturn lkr\n}\n\n// MemIndexAdd adds `nd` to the in memory index.\nfunc (lkr *Linker) MemIndexAdd(nd n.Node, updatePathIndex bool) {\n\tlkr.index[nd.TreeHash().B58String()] = nd\n\tlkr.inodeIndex[nd.Inode()] = nd\n\n\tif updatePathIndex {\n\t\tpath := nd.Path()\n\t\tif nd.Type() == n.NodeTypeDirectory {\n\t\t\tpath = appendDot(path)\n\t\t}\n\t\tlkr.ptrie.InsertWithData(path, nd)\n\t}\n}\n\n// MemIndexSwap updates an entry of the in memory index, by deleting\n// the old entry referenced by oldHash (may be nil). This is necessary\n// to ensure that old hashes do not resolve to the new, updated instance.\n// If the old instance is needed, it will be loaded as new instance.\n// You should not need to call this function, except when implementing own Nodes.\nfunc (lkr *Linker) MemIndexSwap(nd n.Node, oldHash h.Hash, updatePathIndex bool) {\n\tif oldHash != nil {\n\t\tdelete(lkr.index, oldHash.B58String())\n\t}\n\n\tlkr.MemIndexAdd(nd, updatePathIndex)\n}\n\n// MemSetRoot sets the current root, but does not store it yet. It's supposed\n// to be called after in-memory modifications. Only implementors of new Nodes\n// might need to call this function.\nfunc (lkr *Linker) MemSetRoot(root *n.Directory) {\n\tif lkr.root != nil {\n\t\tlkr.MemIndexSwap(root, lkr.root.TreeHash(), true)\n\t} else {\n\t\tlkr.MemIndexAdd(root, true)\n\t}\n\n\tlkr.root = root\n}\n\n// MemIndexPurge removes `nd` from the memory index.\nfunc (lkr *Linker) MemIndexPurge(nd n.Node) {\n\tdelete(lkr.inodeIndex, nd.Inode())\n\tdelete(lkr.index, nd.TreeHash().B58String())\n\tlkr.ptrie.Lookup(nd.Path()).Remove()\n}\n\n// MemIndexClear resets the memory index to zero.\n// This should not be called mid-flight in operations,\n// but should be okay to call between atomic operations.\nfunc (lkr *Linker) MemIndexClear() {\n\tlkr.ptrie = trie.NewNode()\n\tlkr.index = make(map[string]n.Node)\n\tlkr.inodeIndex = make(map[uint64]n.Node)\n\tlkr.root = nil\n}\n\n//////////////////////////\n// COMMON NODE HANDLING //\n//////////////////////////\n\n// NextInode returns a unique identifier, used to identify a single node. You\n// should not need to call this function, except when implementing own nodes.\nfunc (lkr *Linker) NextInode() uint64 {\n\tnodeCount, err := lkr.kv.Get(\"stats\", \"max-inode\")\n\tif err != nil && err != db.ErrNoSuchKey {\n\t\treturn 0\n\t}\n\n\t// nodeCount might be nil on startup:\n\tcnt := uint64(1)\n\tif nodeCount != nil {\n\t\tcnt = binary.BigEndian.Uint64(nodeCount) + 1\n\t}\n\n\tcntBuf := make([]byte, 8)\n\tbinary.BigEndian.PutUint64(cntBuf, cnt)\n\n\terr = lkr.AtomicWithBatch(func(batch db.Batch) (bool, error) {\n\t\tbatch.Put(cntBuf, \"stats\", \"max-inode\")\n\t\treturn false, nil\n\t})\n\n\tif err != nil {\n\t\treturn 0\n\t}\n\n\treturn cnt\n}\n\n// FilesByContents checks what files are associated with the content hashes in\n// `contents`. It returns a map of content hash b58 to file. This method is\n// quite heavy and should not be used in loops. There is room for optimizations.\nfunc (lkr *Linker) FilesByContents(contents []h.Hash) (map[string]*n.File, error) {\n\tkeys, err := lkr.kv.Keys()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tresult := make(map[string]*n.File)\n\tfor _, key := range keys {\n\t\t// Filter non-node storage:\n\t\tfullKey := strings.Join(key, \"/\")\n\t\tif !strings.HasPrefix(fullKey, \"objects\") &&\n\t\t\t!strings.HasPrefix(fullKey, \"stage/objects\") {\n\t\t\tcontinue\n\t\t}\n\n\t\tdata, err := lkr.kv.Get(key...)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tnd, err := n.UnmarshalNode(data)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif nd.Type() != n.NodeTypeFile {\n\t\t\tcontinue\n\t\t}\n\n\t\tfile, ok := nd.(*n.File)\n\t\tif !ok {\n\t\t\treturn nil, ie.ErrBadNode\n\t\t}\n\n\t\tfor _, content := range contents {\n\t\t\tif content.Equal(file.BackendHash()) {\n\t\t\t\tresult[content.B58String()] = file\n\t\t\t}\n\t\t}\n\t}\n\n\treturn result, nil\n}\n\n// loadNode loads an individual object by its hash from the object store. It\n// will return nil if the hash is not there.\nfunc (lkr *Linker) loadNode(hash h.Hash) (n.Node, error) {\n\tvar data []byte\n\tvar err error\n\n\tb58hash := hash.B58String()\n\n\t// First look in the stage:\n\tloadableBuckets := [][]string{\n\t\t{\"stage\", \"objects\", b58hash},\n\t\t{\"objects\", b58hash},\n\t}\n\n\tfor _, bucketPath := range loadableBuckets {\n\t\tdata, err = lkr.kv.Get(bucketPath...)\n\t\tif err != nil && err != db.ErrNoSuchKey {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif data != nil {\n\t\t\treturn n.UnmarshalNode(data)\n\t\t}\n\t}\n\n\t// Damn, no hash found:\n\treturn nil, nil\n}\n\n// NodeByHash returns the node identified by hash.\n// If no such hash could be found, nil is returned.\nfunc (lkr *Linker) NodeByHash(hash h.Hash) (n.Node, error) {\n\t// Check if we have this this node in the memory cache already:\n\tb58Hash := hash.B58String()\n\tif cachedNode, ok := lkr.index[b58Hash]; ok {\n\t\treturn cachedNode, nil\n\t}\n\n\t// Node was not in the cache, load directly from kv.\n\tnd, err := lkr.loadNode(hash)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif nd == nil {\n\t\treturn nil, nil\n\t}\n\n\tlkr.MemIndexAdd(nd, false)\n\treturn nd, nil\n}\n\nfunc appendDot(path string) string {\n\t// path.Join() calls path.Clean() which in turn\n\t// removes the '.' at the end when trying to join that.\n\t// But since we use the dot to mark directories we shouldn't do that.\n\tif strings.HasSuffix(path, \"/\") {\n\t\treturn path + \".\"\n\t}\n\n\treturn path + \"/.\"\n}\n\n// ResolveNode resolves a path to a hash and resolves the corresponding node by\n// calling NodeByHash(). If no node could be resolved, nil is returned.\n// It does not matter if the node was deleted in the meantime. If so,\n// a Ghost node is returned which stores the last known state.\nfunc (lkr *Linker) ResolveNode(nodePath string) (n.Node, error) {\n\t// Check if it's cached already:\n\ttrieNode := lkr.ptrie.Lookup(nodePath)\n\tif trieNode != nil && trieNode.Data != nil {\n\t\treturn trieNode.Data.(n.Node), nil\n\t}\n\n\tfullPaths := [][]string{\n\t\t{\"stage\", \"tree\", nodePath},\n\t\t{\"tree\", nodePath},\n\t}\n\n\tfor _, fullPath := range fullPaths {\n\t\tb58Hash, err := lkr.kv.Get(fullPath...)\n\t\tif err != nil && err != db.ErrNoSuchKey {\n\t\t\treturn nil, e.Wrapf(err, \"db-lookup\")\n\t\t}\n\n\t\tif err == db.ErrNoSuchKey {\n\t\t\tcontinue\n\t\t}\n\n\t\tbhash, err := h.FromB58String(string(b58Hash))\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif bhash != nil {\n\t\t\treturn lkr.NodeByHash(h.Hash(bhash))\n\t\t}\n\t}\n\n\t// Return nil if nothing found:\n\treturn nil, nil\n}\n\n// StageNode inserts a modified node to the staging area, making sure the\n// modification is persistent and part of the staging commit. All parent\n// directories of the node in question will be staged automatically. If there\n// was no modification it will be a (quite expensive) NOOP.\nfunc (lkr *Linker) StageNode(nd n.Node) error {\n\treturn lkr.AtomicWithBatch(func(batch db.Batch) (bool, error) {\n\t\tif err := lkr.stageNodeRecursive(batch, nd); err != nil {\n\t\t\treturn true, e.Wrapf(err, \"recursive stage\")\n\t\t}\n\n\t\t// Update the staging commit's root hash:\n\t\tstatus, err := lkr.Status()\n\t\tif err != nil {\n\t\t\treturn true, fmt.Errorf(\"failed to retrieve status: %v\", err)\n\t\t}\n\n\t\troot, err := lkr.Root()\n\t\tif err != nil {\n\t\t\treturn true, err\n\t\t}\n\n\t\tstatus.SetModTime(time.Now())\n\t\tstatus.SetRoot(root.TreeHash())\n\t\tlkr.MemSetRoot(root)\n\t\treturn hintRollback(lkr.saveStatus(status))\n\t})\n}\n\n// CommitByIndex returns the commit referenced by `index`.\n// `0` will return the very first commit. Negative numbers will yield\n// a ErrNoSuchKey error.\nfunc (lkr *Linker) CommitByIndex(index int64) (*n.Commit, error) {\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif index < 0 {\n\t\t// Interpret an index of -n as curr-(n+1),\n\t\t// so that -1 means \"curr\".\n\t\tindex = status.Index() + index + 1\n\t}\n\n\tb58Hash, err := lkr.kv.Get(\"index\", strconv.FormatInt(index, 10))\n\tif err != nil && err != db.ErrNoSuchKey {\n\t\treturn nil, err\n\t}\n\n\t// Special case: status is not in the index bucket.\n\t// Do a separate check for it.\n\tif err == db.ErrNoSuchKey {\n\t\tif status.Index() == index {\n\t\t\treturn status, nil\n\t\t}\n\n\t\towner, _ := lkr.Owner()\n\t\terrmsg := fmt.Sprintf(\"No commit with index %v for owner `%v`\", index, owner)\n\t\tlog.Error(errmsg)\n\t\treturn nil, ie.NoSuchCommitIndex(index)\n\t}\n\n\thash, err := h.FromB58String(string(b58Hash))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn lkr.CommitByHash(hash)\n}\n\n// NodeByInode resolves a node by it's unique ID.\n// It will return nil if no corresponding node was found.\nfunc (lkr *Linker) NodeByInode(uid uint64) (n.Node, error) {\n\tb58Hash, err := lkr.kv.Get(\"inode\", strconv.FormatUint(uid, 10))\n\tif err != nil && err != db.ErrNoSuchKey {\n\t\treturn nil, err\n\t}\n\n\thash, err := h.FromB58String(string(b58Hash))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn lkr.NodeByHash(hash)\n}\n\nfunc (lkr *Linker) stageNodeRecursive(batch db.Batch, nd n.Node) error {\n\tif nd.Type() == n.NodeTypeCommit {\n\t\treturn fmt.Errorf(\"bug: commits cannot be staged; use MakeCommit()\")\n\t}\n\n\tdata, err := n.MarshalNode(nd)\n\tif err != nil {\n\t\treturn e.Wrapf(err, \"marshal\")\n\t}\n\n\tb58Hash := nd.TreeHash().B58String()\n\tbatch.Put(data, \"stage\", \"objects\", b58Hash)\n\n\tuidKey := strconv.FormatUint(nd.Inode(), 10)\n\tbatch.Put([]byte(nd.TreeHash().B58String()), \"inode\", uidKey)\n\n\thashPath := []string{\"stage\", \"tree\", nd.Path()}\n\tif nd.Type() == n.NodeTypeDirectory {\n\t\thashPath = append(hashPath, \".\")\n\t}\n\n\tbatch.Put([]byte(b58Hash), hashPath...)\n\n\t// Remember/Update this node in the cache if it's not yet there:\n\tlkr.MemIndexAdd(nd, true)\n\n\t// We need to save parent directories too, in case the hash changed:\n\t// Note that this will create many pointless directories in staging.\n\t// That's okay since we garbage collect it every few seconds\n\t// on a higher layer.\n\tif nd.Path() == \"/\" {\n\t\t// Can' go any higher. Save this dir as new virtual root.\n\t\troot, ok := nd.(*n.Directory)\n\t\tif !ok {\n\t\t\treturn ie.ErrBadNode\n\t\t}\n\n\t\tlkr.MemSetRoot(root)\n\t\treturn nil\n\t}\n\n\tpar, err := lkr.ResolveDirectory(path.Dir(nd.Path()))\n\tif err != nil {\n\t\treturn e.Wrapf(err, \"resolve\")\n\t}\n\n\tif par != nil {\n\t\tif err := lkr.stageNodeRecursive(batch, par); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\n/////////////////////\n// COMMIT HANDLING //\n/////////////////////\n\n// SetMergeMarker sets the current status to be a merge commit.\n// Note that this function only will have a result when MakeCommit() is called afterwards.\n// Otherwise, the changes will not be written to disk.\nfunc (lkr *Linker) SetMergeMarker(with string, remoteHead h.Hash) error {\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tstatus.SetMergeMarker(with, remoteHead)\n\treturn lkr.saveStatus(status)\n}\n\n// MakeCommit creates a new full commit in the version history.\n// The current staging commit is finalized with `author` and `message`\n// and gets saved. A new, identical staging commit is created pointing\n// to the root of the now new HEAD.\n//\n// If nothing changed since the last call to MakeCommit, it will\n// return ErrNoChange, which can be reacted upon.\nfunc (lkr *Linker) MakeCommit(author string, message string) error {\n\treturn lkr.AtomicWithBatch(func(batch db.Batch) (bool, error) {\n\t\tswitch err := lkr.makeCommit(batch, author, message); err {\n\t\tcase ie.ErrNoChange:\n\t\t\treturn false, err\n\t\tcase nil:\n\t\t\treturn false, nil\n\t\tdefault:\n\t\t\treturn true, err\n\t\t}\n\t})\n}\n\nfunc (lkr *Linker) makeCommitPutCurrToPersistent(batch db.Batch, rootDir *n.Directory) (map[uint64]bool, error) {\n\texportedInodes := make(map[uint64]bool)\n\treturn exportedInodes, n.Walk(lkr, rootDir, true, func(child n.Node) error {\n\t\tdata, err := n.MarshalNode(child)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tb58Hash := child.TreeHash().B58String()\n\t\tbatch.Put(data, \"objects\", b58Hash)\n\t\texportedInodes[child.Inode()] = true\n\n\t\tchildPath := child.Path()\n\t\tif child.Type() == n.NodeTypeDirectory {\n\t\t\tchildPath = appendDot(childPath)\n\t\t}\n\n\t\tbatch.Put([]byte(b58Hash), \"tree\", childPath)\n\t\treturn nil\n\t})\n}\n\nfunc (lkr *Linker) makeCommit(batch db.Batch, author string, message string) error {\n\thead, err := lkr.Head()\n\tif err != nil && !ie.IsErrNoSuchRef(err) {\n\t\treturn err\n\t}\n\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Only compare with previous if we have a HEAD yet.\n\tif head != nil {\n\t\tif status.Root().Equal(head.Root()) {\n\t\t\treturn ie.ErrNoChange\n\t\t}\n\t}\n\n\trootDir, err := lkr.Root()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Go over all files/directories and save them in tree & objects.\n\t// Note that this will only move nodes that are reachable from the current\n\t// commit root. Intermediate nodes will not be copied.\n\texportedInodes, err := lkr.makeCommitPutCurrToPersistent(batch, rootDir)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// NOTE: `head` may be nil, if it couldn't be resolved,\n\t// or (maybe more likely) if this is the first commit.\n\tif head != nil {\n\t\tif err := status.SetParent(lkr, head); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tif err := status.BoxCommit(author, message); err != nil {\n\t\treturn err\n\t}\n\n\tstatusData, err := n.MarshalNode(status)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tstatusB58Hash := status.TreeHash().B58String()\n\tbatch.Put(statusData, \"objects\", statusB58Hash)\n\n\t// Remember this commit under his index:\n\tbatch.Put([]byte(statusB58Hash), \"index\", strconv.FormatInt(status.Index(), 10))\n\n\tif err := lkr.SaveRef(\"HEAD\", status); err != nil {\n\t\treturn err\n\t}\n\n\t// Check if we have already tagged the initial commit.\n\tif _, err := lkr.ResolveRef(\"init\"); err != nil {\n\t\tif !ie.IsErrNoSuchRef(err) {\n\t\t\t// Some other error happened.\n\t\t\treturn err\n\t\t}\n\n\t\t// This is probably the first commit. Tag it.\n\t\tif err := lkr.SaveRef(\"INIT\", status); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\t// Fixate the moved paths in the stage:\n\tif err := lkr.commitMoveMapping(status, exportedInodes); err != nil {\n\t\treturn err\n\t}\n\n\tif err := lkr.clearStage(batch); err != nil {\n\t\treturn err\n\t}\n\n\tnewStatus, err := n.NewEmptyCommit(lkr.NextInode(), status.Index()+1)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tnewStatus.SetRoot(status.Root())\n\tif err := newStatus.SetParent(lkr, status); err != nil {\n\t\treturn err\n\t}\n\n\treturn lkr.saveStatus(newStatus)\n}\n\nfunc (lkr *Linker) clearStage(batch db.Batch) error {\n\t// Clear the staging area.\n\ttoClear := [][]string{\n\t\t{\"stage\", \"objects\"},\n\t\t{\"stage\", \"tree\"},\n\t\t{\"stage\", \"moves\"},\n\t}\n\n\tfor _, key := range toClear {\n\t\tif err := batch.Clear(key...); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\n///////////////////////\n// METADATA HANDLING //\n///////////////////////\n\n// MetadataPut remembers a value persisntenly identified by `key`.\n// It can be used as single-level key value store for user purposes.\nfunc (lkr *Linker) MetadataPut(key string, value []byte) error {\n\treturn lkr.AtomicWithBatch(func(batch db.Batch) (bool, error) {\n\t\tbatch.Put([]byte(value), \"metadata\", key)\n\t\treturn false, nil\n\t})\n}\n\n// MetadataGet retriesves a previously put key value pair.\n// It will return nil if no such value could be retrieved.\nfunc (lkr *Linker) MetadataGet(key string) ([]byte, error) {\n\treturn lkr.kv.Get(\"metadata\", key)\n}\n\n////////////////////////\n// OWNERSHIP HANDLING //\n////////////////////////\n\n// Owner returns the owner of the linker.\nfunc (lkr *Linker) Owner() (string, error) {\n\tif lkr.owner != \"\" {\n\t\treturn lkr.owner, nil\n\t}\n\n\tdata, err := lkr.MetadataGet(\"owner\")\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\t// Cache owner, we don't want to reload it again and again.\n\t// It will usually not change during runtime, except SetOwner\n\t// is called (which is invalidating the cache anyways)\n\tlkr.owner = string(data)\n\treturn lkr.owner, nil\n}\n\n// SetOwner will set the owner to `owner`.\nfunc (lkr *Linker) SetOwner(owner string) error {\n\tlkr.owner = owner\n\treturn lkr.MetadataPut(\"owner\", []byte(owner))\n}\n\n// SetABIVersion will set the ABI version to `version`.\nfunc (lkr *Linker) SetABIVersion(version int) error {\n\tsv := strconv.Itoa(version)\n\treturn lkr.MetadataPut(\"version\", []byte(sv))\n}\n\n////////////////////////\n// REFERENCE HANDLING //\n////////////////////////\n\n// ResolveRef resolves the hash associated with `refname`. If the ref could not\n// be resolved, ErrNoSuchRef is returned. Typically, Node will be a Commit.\n// But there are no technical restrictions on which node typ to use.\n// NOTE: ResolveRef(\"HEAD\") != ResolveRef(\"head\") due to case.\nfunc (lkr *Linker) ResolveRef(refname string) (n.Node, error) {\n\torigRefname := refname\n\n\tnUps := 0\n\tfor idx := len(refname) - 1; idx >= 0; idx-- {\n\t\tif refname[idx] == '^' {\n\t\t\tnUps++\n\t\t} else {\n\t\t\tbreak\n\t\t}\n\t}\n\n\t// Strip the ^s:\n\trefname = refname[:len(refname)-nUps]\n\n\t// Special case: the status commit is not part of the normal object store.\n\t// Still make it able to resolve it by it's refname \"curr\".\n\tif refname == \"curr\" || refname == \"status\" {\n\t\treturn lkr.Status()\n\t}\n\n\tb58Hash, err := lkr.kv.Get(\"refs\", refname)\n\tif err != nil && err != db.ErrNoSuchKey {\n\t\treturn nil, err\n\t}\n\n\tif len(b58Hash) == 0 {\n\t\t// Try to interpret the refname as b58hash directly.\n\t\t// This path will hit when passing a commit hash directly\n\t\t// as `refname` to this method.\n\t\tb58Hash = []byte(refname)\n\t}\n\n\thash, err := h.FromB58String(string(b58Hash))\n\tif err != nil {\n\t\t// Could not parse hash, so it's probably none.\n\t\treturn nil, ie.ErrNoSuchRef(refname)\n\t}\n\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Special case: Allow the resolving of `curr`\n\t// by using its status hash and check it explicitly.\n\tvar nd n.Node\n\tif status.TreeHash().Equal(hash) {\n\t\tnd = status\n\t} else {\n\t\tnd, err = lkr.NodeByHash(h.Hash(hash))\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tif nd == nil {\n\t\treturn nil, ie.ErrNoSuchRef(refname)\n\t}\n\n\t// Possibly advance a few commits until we hit the one\n\t// the user required.\n\tcmt, ok := nd.(*n.Commit)\n\tif ok {\n\t\tfor i := 0; i < nUps; i++ {\n\t\t\tparentNd, err := cmt.Parent(lkr)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\n\t\t\tif parentNd == nil {\n\t\t\t\tlog.Warningf(\"ref `%s` is too far back; stopping at `init`\", origRefname)\n\t\t\t\tbreak\n\t\t\t}\n\n\t\t\tparentCmt, ok := parentNd.(*n.Commit)\n\t\t\tif !ok {\n\t\t\t\tbreak\n\t\t\t}\n\n\t\t\tcmt = parentCmt\n\t\t}\n\n\t\tnd = cmt\n\t}\n\n\treturn nd, nil\n}\n\n// SaveRef stores a reference to `nd` persistently. The caller is responsbiel\n// to ensure that the node is already in the blockstore, otherwise it won't be\n// resolvable.\nfunc (lkr *Linker) SaveRef(refname string, nd n.Node) error {\n\trefname = strings.ToLower(refname)\n\treturn lkr.AtomicWithBatch(func(batch db.Batch) (bool, error) {\n\t\tbatch.Put([]byte(nd.TreeHash().B58String()), \"refs\", refname)\n\t\treturn false, nil\n\t})\n}\n\n// ListRefs lists all currently known refs.\nfunc (lkr *Linker) ListRefs() ([]string, error) {\n\trefs := []string{}\n\tkeys, err := lkr.kv.Keys(\"refs\")\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor _, key := range keys {\n\t\tif len(key) <= 1 {\n\t\t\tcontinue\n\t\t}\n\n\t\trefs = append(refs, key[1])\n\t}\n\n\treturn refs, nil\n}\n\n// RemoveRef removes the ref named `refname`.\nfunc (lkr *Linker) RemoveRef(refname string) error {\n\treturn lkr.AtomicWithBatch(func(batch db.Batch) (bool, error) {\n\t\tbatch.Erase(\"refs\", refname)\n\t\treturn false, nil\n\t})\n}\n\n// Head is just a shortcut for ResolveRef(\"HEAD\").\nfunc (lkr *Linker) Head() (*n.Commit, error) {\n\tnd, err := lkr.ResolveRef(\"head\")\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcmt, ok := nd.(*n.Commit)\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"oh-oh, HEAD is not a Commit... %v\", nd)\n\t}\n\n\treturn cmt, nil\n}\n\n// Root returns the current root directory of CURR.\n// It is never nil when err is nil.\nfunc (lkr *Linker) Root() (*n.Directory, error) {\n\tif lkr.root != nil {\n\t\treturn lkr.root, nil\n\t}\n\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\trootNd, err := lkr.DirectoryByHash(status.Root())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tlkr.MemSetRoot(rootNd)\n\treturn rootNd, nil\n}\n\n// Status returns the current staging commit.\n// It is never nil, unless err is nil.\nfunc (lkr *Linker) Status() (*n.Commit, error) {\n\tvar cmt *n.Commit\n\tvar err error\n\n\treturn cmt, lkr.AtomicWithBatch(func(batch db.Batch) (bool, error) {\n\t\tcmt, err = lkr.status(batch)\n\t\treturn hintRollback(err)\n\t})\n}\nfunc (lkr *Linker) status(batch db.Batch) (cmt *n.Commit, err error) {\n\tcmt, err = lkr.loadStatus()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif cmt != nil {\n\t\treturn cmt, nil\n\t}\n\n\t// Shoot, no commit exists yet.\n\t// We need to create an initial one.\n\tcmt, err = n.NewEmptyCommit(lkr.NextInode(), 0)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Setup a new commit and set root from last HEAD or new one.\n\thead, err := lkr.Head()\n\tif err != nil && !ie.IsErrNoSuchRef(err) {\n\t\treturn nil, err\n\t}\n\n\tvar rootHash h.Hash\n\n\tif ie.IsErrNoSuchRef(err) {\n\t\t// There probably wasn't a HEAD yet.\n\t\tif root, err := lkr.ResolveDirectory(\"/\"); err == nil && root != nil {\n\t\t\trootHash = root.TreeHash()\n\t\t} else {\n\t\t\t// No root directory then. Create a shiny new one and stage it.\n\t\t\tinode := lkr.NextInode()\n\t\t\tnewRoot, err := n.NewEmptyDirectory(lkr, nil, \"/\", lkr.owner, inode)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\n\t\t\t// Can't call StageNode(), since that would call Status(),\n\t\t\t// causing and endless loop of grief and doom.\n\t\t\tif err := lkr.stageNodeRecursive(batch, newRoot); err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\n\t\t\trootHash = newRoot.TreeHash()\n\t\t}\n\t} else {\n\t\tif err := cmt.SetParent(lkr, head); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\trootHash = head.Root()\n\t}\n\n\tcmt.SetRoot(rootHash)\n\n\tif err := lkr.saveStatus(cmt); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn cmt, nil\n}\n\nfunc (lkr *Linker) loadStatus() (*n.Commit, error) {\n\tdata, err := lkr.kv.Get(\"stage\", \"STATUS\")\n\tif err != nil && err != db.ErrNoSuchKey {\n\t\treturn nil, err\n\t}\n\n\tif data == nil {\n\t\treturn nil, nil\n\t}\n\n\tmsg, err := capnp.Unmarshal(data)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// It's there already. Just unmarshal it.\n\tcmt := &n.Commit{}\n\tif err := cmt.FromCapnp(msg); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn cmt, nil\n}\n\n// saveStatus copies cmt to stage/STATUS.\nfunc (lkr *Linker) saveStatus(cmt *n.Commit) error {\n\treturn lkr.AtomicWithBatch(func(batch db.Batch) (bool, error) {\n\t\thead, err := lkr.Head()\n\t\tif err != nil && !ie.IsErrNoSuchRef(err) {\n\t\t\treturn hintRollback(err)\n\t\t}\n\n\t\tif head != nil {\n\t\t\tif err := cmt.SetParent(lkr, head); err != nil {\n\t\t\t\treturn hintRollback(err)\n\t\t\t}\n\t\t}\n\n\t\tif err := cmt.BoxCommit(n.AuthorOfStage, \"\"); err != nil {\n\t\t\treturn hintRollback(err)\n\t\t}\n\n\t\tdata, err := n.MarshalNode(cmt)\n\t\tif err != nil {\n\t\t\treturn hintRollback(err)\n\t\t}\n\n\t\tinode := strconv.FormatUint(cmt.Inode(), 10)\n\t\tbatch.Put(data, \"stage\", \"STATUS\")\n\t\tbatch.Put([]byte(cmt.TreeHash().B58String()), \"inode\", inode)\n\t\treturn hintRollback(lkr.SaveRef(\"CURR\", cmt))\n\t})\n}\n\n/////////////////////////////////\n// CONVINIENT ACCESS FUNCTIONS //\n/////////////////////////////////\n\n// LookupNode takes the root node and tries to resolve the path from there.\n// Deleted paths are recognized in contrast to ResolveNode.\n// If a path does not exist NoSuchFile is returned.\nfunc (lkr *Linker) LookupNode(repoPath string) (n.Node, error) {\n\troot, err := lkr.Root()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn root.Lookup(lkr, repoPath)\n}\n\n// LookupNodeAt works like LookupNode but returns the node at the state of `cmt`.\nfunc (lkr *Linker) LookupNodeAt(cmt *n.Commit, repoPath string) (n.Node, error) {\n\troot, err := lkr.DirectoryByHash(cmt.Root())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif root == nil {\n\t\treturn nil, nil\n\t}\n\n\treturn root.Lookup(lkr, repoPath)\n}\n\n// LookupModNode is like LookupNode but returns a readily cast ModNode.\nfunc (lkr *Linker) LookupModNode(repoPath string) (n.ModNode, error) {\n\tnode, err := lkr.LookupNode(repoPath)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif node == nil {\n\t\treturn nil, nil\n\t}\n\n\tsnode, ok := node.(n.ModNode)\n\tif !ok {\n\t\treturn nil, ie.ErrBadNode\n\t}\n\n\treturn snode, nil\n}\n\n// LookupModNodeAt is like LookupNodeAt but with readily cast type.\nfunc (lkr *Linker) LookupModNodeAt(cmt *n.Commit, repoPath string) (n.ModNode, error) {\n\tnode, err := lkr.LookupNodeAt(cmt, repoPath)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif node == nil {\n\t\treturn nil, nil\n\t}\n\n\tsnode, ok := node.(n.ModNode)\n\tif !ok {\n\t\treturn nil, ie.ErrBadNode\n\t}\n\n\treturn snode, nil\n}\n\n// DirectoryByHash calls NodeByHash and attempts to convert\n// it to a Directory as convinience.\nfunc (lkr *Linker) DirectoryByHash(hash h.Hash) (*n.Directory, error) {\n\tnd, err := lkr.NodeByHash(hash)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif nd == nil {\n\t\treturn nil, nil\n\t}\n\n\tdir, ok := nd.(*n.Directory)\n\tif !ok {\n\t\treturn nil, ie.ErrBadNode\n\t}\n\n\treturn dir, nil\n}\n\n// ResolveDirectory calls ResolveNode and converts the result to a Directory.\n// This only accesses nodes from the filesystem and does not differentiate\n// between ghosts and living nodes.\nfunc (lkr *Linker) ResolveDirectory(dirpath string) (*n.Directory, error) {\n\tnd, err := lkr.ResolveNode(appendDot(path.Clean(dirpath)))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif nd == nil {\n\t\treturn nil, nil\n\t}\n\n\tdir, ok := nd.(*n.Directory)\n\tif !ok {\n\t\treturn nil, ie.ErrBadNode\n\t}\n\n\treturn dir, nil\n}\n\n// LookupDirectory calls LookupNode and converts the result to a Directory.\nfunc (lkr *Linker) LookupDirectory(repoPath string) (*n.Directory, error) {\n\tnd, err := lkr.LookupNode(repoPath)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif nd == nil {\n\t\treturn nil, nil\n\t}\n\n\tdir, ok := nd.(*n.Directory)\n\tif !ok {\n\t\treturn nil, ie.ErrBadNode\n\t}\n\n\treturn dir, nil\n}\n\n// FileByHash calls NodeByHash and converts the result to a File.\nfunc (lkr *Linker) FileByHash(hash h.Hash) (*n.File, error) {\n\tnd, err := lkr.NodeByHash(hash)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfile, ok := nd.(*n.File)\n\tif !ok {\n\t\treturn nil, ie.ErrBadNode\n\t}\n\n\treturn file, nil\n}\n\n// LookupFile calls LookupNode and converts the result to a file.\nfunc (lkr *Linker) LookupFile(repoPath string) (*n.File, error) {\n\tnd, err := lkr.LookupNode(repoPath)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif nd == nil {\n\t\treturn nil, nil\n\t}\n\n\tfile, ok := nd.(*n.File)\n\tif !ok {\n\t\treturn nil, ie.ErrBadNode\n\t}\n\n\treturn file, nil\n}\n\n// LookupGhost calls LookupNode and converts the result to a ghost.\nfunc (lkr *Linker) LookupGhost(repoPath string) (*n.Ghost, error) {\n\tnd, err := lkr.LookupNode(repoPath)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif nd == nil {\n\t\treturn nil, nil\n\t}\n\n\tghost, ok := nd.(*n.Ghost)\n\tif !ok {\n\t\treturn nil, ie.ErrBadNode\n\t}\n\n\treturn ghost, nil\n}\n\n// CommitByHash lookups a commit by it's hash.\n// If the commit could not be found, nil is returned.\nfunc (lkr *Linker) CommitByHash(hash h.Hash) (*n.Commit, error) {\n\tnd, err := lkr.NodeByHash(hash)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif nd == nil {\n\t\treturn nil, nil\n\t}\n\n\tcmt, ok := nd.(*n.Commit)\n\tif !ok {\n\t\treturn nil, ie.ErrBadNode\n\t}\n\n\treturn cmt, nil\n}\n\n// HaveStagedChanges returns true if there were changes in the staging area.\n// If an error occurs, the first return value is undefined.\nfunc (lkr *Linker) HaveStagedChanges() (bool, error) {\n\thead, err := lkr.Head()\n\tif err != nil && !ie.IsErrNoSuchRef(err) {\n\t\treturn false, err\n\t}\n\n\tif ie.IsErrNoSuchRef(err) {\n\t\t// There is no HEAD yet. Assume we have changes.\n\t\treturn true, nil\n\t}\n\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\t// Check if the root hashes of CURR and HEAD differ.\n\treturn !status.Root().Equal(head.Root()), nil\n}\n\n// CheckoutCommit resets the current staging commit back to the commit\n// referenced by cmt. If force is false, it will check if there any staged errors in\n// the staging area and return ErrStageNotEmpty if there are any. If force is\n// true, all changes will be overwritten.\nfunc (lkr *Linker) CheckoutCommit(cmt *n.Commit, force bool) error {\n\t// Check if the staging area is empty if no force given:\n\tif !force {\n\t\thaveStaged, err := lkr.HaveStagedChanges()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif haveStaged {\n\t\t\treturn ie.ErrStageNotEmpty\n\t\t}\n\t}\n\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\troot, err := lkr.DirectoryByHash(cmt.Root())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn lkr.Atomic(func() (bool, error) {\n\t\t// Set the current virtual in-memory cached root\n\t\tlkr.MemSetRoot(root)\n\t\tstatus.SetRoot(cmt.Root())\n\n\t\t// Invalidate the cache, causing NodeByHash and ResolveNode to load the\n\t\t// file from the boltdb again:\n\t\tlkr.MemIndexClear()\n\t\treturn hintRollback(lkr.saveStatus(status))\n\t})\n}\n\n// AddMoveMapping takes note that the the node with `fromInode` has been moved\n// to `toInode` in the staging commit.\nfunc (lkr *Linker) AddMoveMapping(fromInode, toInode uint64) (err error) {\n\t// Make sure the actual checkout will land as one batch on disk:\n\tsrcInode := strconv.FormatUint(fromInode, 10)\n\tsrcToDstKey := []string{\"stage\", \"moves\", srcInode}\n\n\tdstInode := strconv.FormatUint(toInode, 10)\n\tdstToSrcKey := []string{\"stage\", \"moves\", dstInode}\n\n\treturn lkr.AtomicWithBatch(func(batch db.Batch) (bool, error) {\n\t\tif _, err = lkr.kv.Get(srcToDstKey...); err == db.ErrNoSuchKey {\n\t\t\tline := []byte(fmt.Sprintf(\"> inode %d\", toInode))\n\t\t\tbatch.Put(line, srcToDstKey...)\n\t\t\tbatch.Put(line, \"stage\", \"moves\", \"overlay\", srcInode)\n\t\t} else if err != nil {\n\t\t\treturn hintRollback(err)\n\t\t}\n\n\t\t// Also remember the move in the other direction.\n\t\tif _, err = lkr.kv.Get(dstToSrcKey...); err == db.ErrNoSuchKey {\n\t\t\tline := []byte(fmt.Sprintf(\"< inode %d\", fromInode))\n\t\t\tbatch.Put(line, dstToSrcKey...)\n\t\t\tbatch.Put(line, \"stage\", \"moves\", \"overlay\", dstInode)\n\t\t} else {\n\t\t\treturn hintRollback(err)\n\t\t}\n\n\t\treturn false, nil\n\t})\n}\n\nfunc (lkr *Linker) parseMoveMappingLine(line string) (n.Node, MoveDir, error) {\n\tsplitLine := strings.SplitN(line, \" \", 3)\n\tif len(splitLine) < 3 {\n\t\treturn nil, 0, fmt.Errorf(\"Malformed stage move line: `%s`\", line)\n\t}\n\n\tdir := moveDirFromString(splitLine[0])\n\tif dir == MoveDirUnknown {\n\t\treturn nil, 0, fmt.Errorf(\"Unrecognized move direction `%s`\", splitLine[0])\n\t}\n\n\tswitch splitLine[1] {\n\tcase \"inode\":\n\t\tinode, err := strconv.ParseUint(splitLine[2], 10, 64)\n\t\tif err != nil {\n\t\t\treturn nil, 0, err\n\t\t}\n\n\t\tnode, err := lkr.NodeByInode(inode)\n\t\tif err != nil {\n\t\t\treturn nil, 0, err\n\t\t}\n\n\t\treturn node, dir, nil\n\tcase \"hash\":\n\t\thash, err := h.FromB58String(splitLine[2])\n\t\tif err != nil {\n\t\t\treturn nil, 0, err\n\t\t}\n\n\t\tnode, err := lkr.NodeByHash(hash)\n\t\tif err != nil {\n\t\t\treturn nil, 0, err\n\t\t}\n\n\t\treturn node, dir, nil\n\tdefault:\n\t\treturn nil, 0, fmt.Errorf(\"Unsupported move map type: %s\", splitLine[1])\n\t}\n}\n\n// Process a sinlge key of the move mapping:\nfunc (lkr *Linker) commitMoveMappingKey(\n\tbatch db.Batch,\n\tstatus *n.Commit,\n\texported map[uint64]bool,\n\tkey []string,\n) error {\n\tinode, err := strconv.ParseUint(key[len(key)-1], 10, 64)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Only export move mapping that relate to nodes that were actually\n\t// exported from staging. We do not want to export intermediate moves.\n\tif _, ok := exported[inode]; !ok {\n\t\treturn nil\n\t}\n\n\tdata, err := lkr.kv.Get(key...)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdstNode, moveDirection, err := lkr.parseMoveMappingLine(string(data))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif moveDirection == MoveDirDstToSrc {\n\t\treturn nil\n\t}\n\n\tif dstNode == nil {\n\t\treturn fmt.Errorf(\"Failed to find dest node for commit map: %v\", string(data))\n\t}\n\n\tsrcNode, err := lkr.NodeByInode(inode)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif srcNode == nil {\n\t\treturn fmt.Errorf(\"Failed to find source node for commit map: %d\", inode)\n\t}\n\n\t// Write a bidirectional mapping for this node:\n\tdstB58 := dstNode.TreeHash().B58String()\n\tsrcB58 := srcNode.TreeHash().B58String()\n\n\tforwardLine := fmt.Sprintf(\"%v hash %s\", moveDirection, dstB58)\n\tbatch.Put(\n\t\t[]byte(forwardLine),\n\t\t\"moves\", status.TreeHash().B58String(), srcB58,\n\t)\n\n\tbatch.Put(\n\t\t[]byte(forwardLine),\n\t\t\"moves\", \"overlay\", srcB58,\n\t)\n\n\treverseLine := fmt.Sprintf(\n\t\t\"%v hash %s\",\n\t\tmoveDirection.Invert(),\n\t\tsrcB58,\n\t)\n\n\tbatch.Put(\n\t\t[]byte(reverseLine),\n\t\t\"moves\", status.TreeHash().B58String(), dstB58,\n\t)\n\n\tbatch.Put(\n\t\t[]byte(reverseLine),\n\t\t\"moves\", \"overlay\", dstB58,\n\t)\n\n\t// We need to verify that all ghosts will be copied out from staging.\n\t// In some special cases, not all used ghosts are reachable in\n\t// MakeCommit.\n\t//\n\t// Consider for example this case:\n\t//\n\t// $ touch x\n\t// $ commit\n\t// $ move x y\n\t// $ touch x\n\t// $ commit\n\t//\n\t// => In the last commit the ghost from the move (x) is overwritten by\n\t// a new file and thus will not be reachable anymore. In order to store\n\t// the full history of the file we need to also keep this ghost.\n\tfor _, checkHash := range []string{dstB58, srcB58} {\n\t\tsrcKey := []string{\"stage\", \"objects\", checkHash}\n\t\tdstKey := []string{\"objects\", checkHash}\n\n\t\t_, err = lkr.kv.Get(dstKey...)\n\t\tif err == db.ErrNoSuchKey {\n\t\t\terr = nil\n\n\t\t\t// This part of the move was not reachable, we need to copy it\n\t\t\t// to the object store additionally.\n\t\t\tif err := db.CopyKey(lkr.kv, srcKey, dstKey); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\t// We already have a bidir mapping for this node, no need to mention\n\t// them further. (would not hurt, but would be duplicated work)\n\tdelete(exported, srcNode.Inode())\n\tdelete(exported, dstNode.Inode())\n\treturn nil\n}\n\nfunc (lkr *Linker) commitMoveMapping(status *n.Commit, exported map[uint64]bool) error {\n\treturn lkr.AtomicWithBatch(func(batch db.Batch) (bool, error) {\n\t\tkeys, err := lkr.kv.Keys(\"stage\", \"moves\")\n\t\tif err != nil {\n\t\t\treturn hintRollback(err)\n\t\t}\n\n\t\tfor _, key := range keys {\n\t\t\tif err := lkr.commitMoveMappingKey(batch, status, exported, key); err != nil {\n\t\t\t\treturn hintRollback(err)\n\t\t\t}\n\t\t}\n\n\t\treturn false, nil\n\t})\n}\n\nconst (\n\t// MoveDirUnknown should only be used for init purposes.\n\tMoveDirUnknown = iota\n\t// MoveDirSrcToDst means that this file was moved from source to dest.\n\t// (Therefore it is the new destination file and probably not a ghost)\n\tMoveDirSrcToDst\n\t// MoveDirDstToSrc means that this place was moved somewhere else.\n\t// (Therefore it is a likely a ghost and the new file lives somewhere else)\n\tMoveDirDstToSrc\n\t// MoveDirNone tells us that this file did not move.\n\tMoveDirNone\n)\n\n// MoveDir describes the direction of a move.\ntype MoveDir int\n\nfunc (md MoveDir) String() string {\n\tswitch md {\n\tcase MoveDirSrcToDst:\n\t\treturn \">\"\n\tcase MoveDirDstToSrc:\n\t\treturn \"<\"\n\tcase MoveDirNone:\n\t\treturn \"*\"\n\tdefault:\n\t\treturn \"\"\n\t}\n}\n\n// Invert changes the direction of a move, if it has one.\nfunc (md MoveDir) Invert() MoveDir {\n\tswitch md {\n\tcase MoveDirSrcToDst:\n\t\treturn MoveDirDstToSrc\n\tcase MoveDirDstToSrc:\n\t\treturn MoveDirSrcToDst\n\tdefault:\n\t\treturn md\n\t}\n}\n\nfunc moveDirFromString(spec string) MoveDir {\n\tswitch spec {\n\tcase \">\":\n\t\treturn MoveDirSrcToDst\n\tcase \"<\":\n\t\treturn MoveDirDstToSrc\n\tcase \"*\":\n\t\treturn MoveDirNone\n\tdefault:\n\t\treturn MoveDirUnknown\n\t}\n}\n\n// MoveEntryPoint tells us if a node participated in a move.\n// If so, the new node and the corresponding move direction is returned.\nfunc (lkr *Linker) MoveEntryPoint(nd n.Node) (n.Node, MoveDir, error) {\n\tmoveData, err := lkr.kv.Get(\n\t\t\"stage\", \"moves\", \"overlay\",\n\t\tstrconv.FormatUint(nd.Inode(), 10),\n\t)\n\n\tif err != nil && err != db.ErrNoSuchKey {\n\t\treturn nil, MoveDirUnknown, err\n\t}\n\n\tif moveData == nil {\n\t\tmoveData, err = lkr.kv.Get(\"moves\", \"overlay\", nd.TreeHash().B58String())\n\t\tif err != nil && err != db.ErrNoSuchKey {\n\t\t\treturn nil, MoveDirUnknown, err\n\t\t}\n\n\t\tif moveData == nil {\n\t\t\treturn nil, MoveDirNone, nil\n\t\t}\n\t}\n\n\tnode, moveDir, err := lkr.parseMoveMappingLine(string(moveData))\n\tif err != nil {\n\t\treturn nil, MoveDirUnknown, err\n\t}\n\n\tif node == nil {\n\t\t// No move mapping found for this node.\n\t\t// Note that this not an error.\n\t\treturn nil, MoveDirNone, nil\n\t}\n\n\treturn node, moveDir, err\n}\n\n// MoveMapping will lookup if the node pointed to by `nd` was part of a moving\n// operation and if so, to what node it was moved and if it was the source or\n// the dest node.\nfunc (lkr *Linker) MoveMapping(cmt *n.Commit, nd n.Node) (n.Node, MoveDir, error) {\n\t// Stage and committed space use a different format to store move mappings.\n\t// This is because in staging nodes can still be modified, so the \"dest\"\n\t// part of the mapping is a moving target. Therefore we store the destination\n\t// not as hash or path (which also might be moved), but as inode reference.\n\t// Inodes always resolve to the latest version of a node.\n\t// When committing, the mappings will be \"fixed\" by converting the inode to\n\t// a hash value, to make sure we link to a specific version.\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\treturn nil, MoveDirUnknown, err\n\t}\n\n\t// Only look into staging if we are actually in the STATUS commit.\n\t// The lookups in the stage level are on an inode base. This would\n\t// cause jumping around in the history for older commits.\n\tif cmt == nil || cmt.TreeHash().Equal(status.TreeHash()) {\n\t\tinodeKey := strconv.FormatUint(nd.Inode(), 10)\n\t\tmoveData, err := lkr.kv.Get(\"stage\", \"moves\", inodeKey)\n\t\tif err != nil && err != db.ErrNoSuchKey {\n\t\t\treturn nil, MoveDirUnknown, err\n\t\t}\n\n\t\tif err != db.ErrNoSuchKey {\n\t\t\tnode, moveDir, err := lkr.parseMoveMappingLine(string(moveData))\n\t\t\tif err != nil {\n\t\t\t\treturn nil, MoveDirUnknown, err\n\t\t\t}\n\n\t\t\tif node != nil {\n\t\t\t\treturn node, moveDir, err\n\t\t\t}\n\t\t}\n\t}\n\n\tif cmt == nil {\n\t\treturn nil, MoveDirNone, nil\n\t}\n\n\tmoveData, err := lkr.kv.Get(\"moves\", cmt.TreeHash().B58String(), nd.TreeHash().B58String())\n\tif err != nil && err != db.ErrNoSuchKey {\n\t\treturn nil, MoveDirUnknown, err\n\t}\n\n\tif moveData == nil {\n\t\treturn nil, MoveDirNone, nil\n\t}\n\n\tnode, moveDir, err := lkr.parseMoveMappingLine(string(moveData))\n\tif err != nil {\n\t\treturn nil, MoveDirUnknown, err\n\t}\n\n\tif node == nil {\n\t\t// No move mapping found for this node.\n\t\t// Note that this not an error.\n\t\treturn nil, MoveDirNone, nil\n\t}\n\n\treturn node, moveDir, err\n}\n\n// ExpandAbbrev tries to find an object reference that stats with `abbrev`.\n// If so, it will return the respective hash for it.\n// If none is found, it is considered as an error.\n// If more than one was found ie.ErrAmbigious is returned.\nfunc (lkr *Linker) ExpandAbbrev(abbrev string) (h.Hash, error) {\n\tprefixes := [][]string{\n\t\t{\"stage\", \"objects\"},\n\t\t{\"objects\"},\n\t}\n\n\t// Special case: Make it possible to abbrev the commit\n\t// of ``curr`` - it does live in stage/STATUS, not somewhere else.\n\tcurr, err := lkr.Status()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif strings.HasPrefix(curr.TreeHash().B58String(), abbrev) {\n\t\treturn curr.TreeHash(), nil\n\t}\n\n\tfor _, prefix := range prefixes {\n\t\tmatches, err := lkr.kv.Glob(append(prefix, abbrev))\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif len(matches) > 1 {\n\t\t\treturn nil, ie.ErrAmbigiousRev\n\t\t}\n\n\t\tif len(matches) == 0 {\n\t\t\tcontinue\n\t\t}\n\n\t\tmatch := matches[0]\n\t\treturn h.FromB58String(match[len(match)-1])\n\t}\n\n\treturn nil, fmt.Errorf(\"No such abbrev: %v\", abbrev)\n}\n\n// IterAll goes over all nodes in the commit range `from` until (including) `to`.\n// Already visited nodes will not be visited again if they did not change.\n// If `from` is nil, HEAD is assumed.\n// If `to` is nil, INIT is assumed.\nfunc (lkr *Linker) IterAll(from, to *n.Commit, fn func(n.ModNode, *n.Commit) error) error {\n\tvisited := make(map[string]struct{})\n\treturn lkr.iterAll(from, to, visited, fn)\n}\n\nfunc (lkr *Linker) iterAll(from, to *n.Commit, visited map[string]struct{}, fn func(n.ModNode, *n.Commit) error) error {\n\tif from == nil {\n\t\thead, err := lkr.Status()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfrom = head\n\t}\n\n\troot, err := lkr.DirectoryByHash(from.Root())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\twalker := func(child n.Node) error {\n\t\tif _, ok := visited[child.TreeHash().B58String()]; ok {\n\t\t\treturn n.ErrSkipChild\n\t\t}\n\n\t\tmodChild, ok := child.(n.ModNode)\n\t\tif !ok {\n\t\t\treturn ie.ErrBadNode\n\t\t}\n\n\t\tvisited[child.TreeHash().B58String()] = struct{}{}\n\t\treturn fn(modChild, from)\n\t}\n\n\tif err := n.Walk(lkr, root, false, walker); err != nil {\n\t\treturn e.Wrapf(err, \"iter-all: walk\")\n\t}\n\n\t// Check if we're already at the lowest commit:\n\tif to != nil && from.TreeHash().Equal(to.TreeHash()) {\n\t\treturn nil\n\t}\n\n\tprev, err := from.Parent(lkr)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif prev == nil {\n\t\t// Definite end of line.\n\t\treturn nil\n\t}\n\n\tprevCmt, ok := prev.(*n.Commit)\n\tif !ok {\n\t\treturn ie.ErrBadNode\n\t}\n\n\treturn lkr.iterAll(prevCmt, to, visited, fn)\n}\n\n// Atomic is like AtomicWithBatch but does not require using a batch.\n// Use this for read-only operations. It's only syntactic sugar though.\nfunc (lkr *Linker) Atomic(fn func() (bool, error)) (err error) {\n\treturn lkr.AtomicWithBatch(func(batch db.Batch) (bool, error) {\n\t\treturn fn()\n\t})\n}\n\n// AtomicWithBatch will execute `fn` in one transaction.\n// If anything goes wrong (i.e. `fn` returns an error)\nfunc (lkr *Linker) AtomicWithBatch(fn func(batch db.Batch) (bool, error)) (err error) {\n\tbatch := lkr.kv.Batch()\n\n\t// A panicking program should not leave the persistent linker state\n\t// inconsistent. This is really a last defence against all odds.\n\tdefer func() {\n\t\tif r := recover(); r != nil {\n\t\t\tbatch.Rollback()\n\t\t\tlkr.MemIndexClear()\n\t\t\terr = fmt.Errorf(\"panic rollback: %v; stack: %s\", r, string(debug.Stack()))\n\t\t}\n\t}()\n\n\tneedRollback, err := fn(batch)\n\tif needRollback && err != nil {\n\t\thadWrites := batch.HaveWrites()\n\t\tbatch.Rollback()\n\n\t\t// Only clear the whole index if something was written.\n\t\t// Also, this prevents the slightly misleading log message below\n\t\t// in case of read-only operations.\n\t\tif hadWrites {\n\t\t\t// clearing the mem index will cause it to be read freshly from disk\n\t\t\t// with the old state. This costs a little performance but saves me\n\t\t\t// from writing special in-memory rollback logic for now.\n\t\t\tlkr.MemIndexClear()\n\t\t\tlog.Warningf(\"rolled back due to error: %v %s\", err, debug.Stack())\n\t\t}\n\n\t\treturn err\n\t}\n\n\t// Attempt to write it to disk.\n\t// If that fails we're better off deleting our internal cache.\n\t// so memory and disk is in sync.\n\tif flushErr := batch.Flush(); flushErr != nil {\n\t\tlkr.MemIndexClear()\n\t\tlog.Warningf(\"flush to db failed, resetting mem index: %v\", flushErr)\n\t}\n\n\treturn err\n}\n\n// helper to return errros that should trigger a rollback in AtomicWithBatch()\nfunc hintRollback(err error) (bool, error) {\n\tif err != nil {\n\t\treturn true, err\n\t}\n\n\treturn false, nil\n}\n\n// KV returns the key value store passed when constructing the linker.\nfunc (lkr *Linker) KV() db.Database {\n\treturn lkr.kv\n}\n" }, { "path": "catfs/core/linker_test.go", "content": "package core\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"sort\"\n\t\"strings\"\n\t\"testing\"\n\t\"unsafe\"\n\n\t\"github.com/sahib/brig/catfs/db\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// Basic test to see if the root node can be inserted and stored.\n// A new staging commit should be also created in the background.\n// On the second run, the root node should be already cached.\nfunc TestLinkerInsertRoot(t *testing.T) {\n\tWithDummyKv(t, func(kv db.Database) {\n\t\tlkr := NewLinker(kv)\n\t\troot, err := n.NewEmptyDirectory(lkr, nil, \"/\", \"u\", 2)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Creating empty root dir failed: %v\", err)\n\t\t}\n\n\t\tif err := lkr.StageNode(root); err != nil {\n\t\t\tt.Fatalf(\"Staging root failed: %v\", err)\n\t\t}\n\n\t\tsameRoot, err := lkr.ResolveDirectory(\"/\")\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Resolving root failed: %v\", err)\n\t\t}\n\n\t\tif sameRoot == nil {\n\t\t\tt.Fatal(\"Resolving root failed (is nil)\")\n\t\t}\n\n\t\tif path := sameRoot.Path(); path != \"/\" {\n\t\t\tt.Fatalf(\"Path of root is not /: %s\", path)\n\t\t}\n\n\t\tptrRoot, err := lkr.ResolveDirectory(\"/\")\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Second lookup of root failed: %v\", err)\n\t\t}\n\n\t\tif unsafe.Pointer(ptrRoot) != unsafe.Pointer(sameRoot) {\n\t\t\tt.Fatal(\"Second root did not come from the cache\")\n\t\t}\n\n\t\tstatus, err := lkr.Status()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to retrieve status: %v\", err)\n\t\t}\n\n\t\tif !status.Root().Equal(root.TreeHash()) {\n\t\t\tt.Fatalf(\"status.root and root differ: %v <-> %v\", status.Root(), root.TreeHash())\n\t\t}\n\t})\n}\n\nfunc TestLinkerRefs(t *testing.T) {\n\tauthor := n.AuthorOfStage\n\tWithDummyKv(t, func(kv db.Database) {\n\t\tlkr := NewLinker(kv)\n\t\troot, err := lkr.Root()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to create root: %v\", err)\n\t\t}\n\n\t\tnewFile := n.NewEmptyFile(root, \"cat.png\", \"u\", 2)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to create empty file: %v\", err)\n\t\t}\n\n\t\tnewFile.SetSize(10)\n\t\tnewFile.SetContent(lkr, h.TestDummy(t, 1))\n\n\t\tif err := root.Add(lkr, newFile); err != nil {\n\t\t\tt.Fatalf(\"Adding empty file failed: %v\", err)\n\t\t}\n\n\t\tif err := lkr.StageNode(newFile); err != nil {\n\t\t\tt.Fatalf(\"Staging new file failed: %v\", err)\n\t\t}\n\n\t\tif _, err := lkr.Head(); !ie.IsErrNoSuchRef(err) {\n\t\t\tt.Fatalf(\"There is a HEAD from start?!\")\n\t\t}\n\n\t\tcmt, err := lkr.Status()\n\t\tif err != nil || cmt == nil {\n\t\t\tt.Fatalf(\"Failed to retrieve status: %v\", err)\n\t\t}\n\n\t\tif err := lkr.MakeCommit(author, \"First commit\"); err != nil {\n\t\t\tt.Fatalf(\"Making commit failed: %v\", err)\n\t\t}\n\n\t\t// Assert that staging is empy (except the \"/stage/STATUS\" part)\n\t\tfoundKeys := []string{}\n\t\tkeys, err := kv.Keys(\"stage\")\n\t\trequire.Nil(t, err)\n\n\t\tfor _, key := range keys {\n\t\t\tfoundKeys = append(foundKeys, strings.Join(key, \"/\"))\n\t\t}\n\n\t\trequire.Equal(t, []string{\"stage/STATUS\"}, foundKeys)\n\n\t\thead, err := lkr.Head()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Obtaining HEAD failed: %v\", err)\n\t\t}\n\n\t\tstatus, err := lkr.Status()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to obtain the status: %v\", err)\n\t\t}\n\n\t\tif !head.Root().Equal(status.Root()) {\n\t\t\tt.Fatalf(\"HEAD and CURR are not equal after first commit.\")\n\t\t}\n\n\t\tif err := lkr.MakeCommit(author, \"No.\"); err != ie.ErrNoChange {\n\t\t\tt.Fatalf(\"Committing without change led to a new commit: %v\", err)\n\t\t}\n\t})\n}\n\nfunc TestLinkerNested(t *testing.T) {\n\tWithDummyKv(t, func(kv db.Database) {\n\t\tlkr := NewLinker(kv)\n\t\troot, err := lkr.Root()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Fetching initial root failed: %v\", err)\n\t\t\treturn\n\t\t}\n\n\t\tsub, err := n.NewEmptyDirectory(lkr, root, \"sub\", \"u\", 3)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Creating empty sub dir failed: %v\", err)\n\t\t\treturn\n\t\t}\n\n\t\tpar, err := sub.Parent(lkr)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to get parent of /sub\")\n\t\t}\n\n\t\tif par.Path() != \"/\" {\n\t\t\tt.Fatalf(\"Parent path of /sub is not /\")\n\t\t}\n\n\t\tif topPar, err := par.Parent(lkr); topPar != nil || err != nil {\n\t\t\tt.Fatalf(\"Parent of / is not nil: %v (%v)\", topPar, err)\n\t\t}\n\n\t\tif err := lkr.StageNode(sub); err != nil {\n\t\t\tt.Fatalf(\"Staging /sub failed: %v\", err)\n\t\t}\n\n\t\tsameSubDir, err := lkr.ResolveDirectory(\"/sub\")\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Resolving /sub failed: %v\", err)\n\t\t}\n\n\t\t_, err = lkr.NodeByInode(sameSubDir.Inode())\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Resolving /sub by ID (%d) failed: %v\", sameSubDir.Inode(), err)\n\t\t}\n\n\t\tsubpub, err := n.NewEmptyDirectory(lkr, sameSubDir, \"pub\", \"u\", 4)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Creating of deep sub failed\")\n\t\t}\n\n\t\tif err := lkr.StageNode(subpub); err != nil {\n\t\t\tt.Fatalf(\"Staging /sub/pub failed: %v\", err)\n\t\t}\n\n\t\tnewRootDir, err := lkr.ResolveDirectory(\"/\")\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to resolve new root dir\")\n\t\t}\n\n\t\tif !newRootDir.TreeHash().Equal(root.TreeHash()) {\n\t\t\tt.Fatalf(\"New / and old / have different hashes, despite being same instance %p %p\", newRootDir, root)\n\t\t}\n\n\t\tcount := 0\n\t\tif err := n.Walk(lkr, root, true, func(c n.Node) error { count++; return nil }); err != nil {\n\t\t\tt.Fatalf(\"Failed to walk the tree: %v\", err)\n\t\t}\n\n\t\tif count != 3 {\n\t\t\tt.Fatalf(\"There are more or less than 3 elems in the tree: %d\", count)\n\t\t}\n\n\t\t// Index shall only contain the nodes with their most current hash values.\n\t\tif len(lkr.index) != 3 {\n\t\t\tt.Fatalf(\"Index does not contain the expected 3 elements.\")\n\t\t}\n\n\t\tgc := NewGarbageCollector(lkr, kv, nil)\n\t\tif err := gc.Run(true); err != nil {\n\t\t\tt.Fatalf(\"Garbage collector failed to run: %v\", err)\n\t\t}\n\n\t\tif err := lkr.MakeCommit(n.AuthorOfStage, \"first message\"); err != nil {\n\t\t\tt.Fatalf(\"Making first commit failed: %v\", err)\n\t\t}\n\t})\n}\n\n// Test if Linker can load objects after closing/re-opening the kv.\nfunc TestLinkerPersistence(t *testing.T) {\n\tdbPath, err := ioutil.TempDir(\"\", \"brig-test\")\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to create temp dir: %v\", err)\n\t}\n\n\tdefer os.RemoveAll(dbPath)\n\n\tkv, err := db.NewDiskDatabase(dbPath)\n\tif err != nil {\n\t\tt.Fatalf(\"Could not create dummy kv for tests: %v\", err)\n\t}\n\n\tlkr := NewLinker(kv)\n\tif err := lkr.MakeCommit(n.AuthorOfStage, \"initial commit\"); err != nil {\n\t\tt.Fatalf(\"Failed to create initial commit out of nothing: %v\", err)\n\t}\n\n\thead, err := lkr.Head()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to retrieve Head after initial commit: %v\", err)\n\t}\n\n\toldHeadHash := head.TreeHash().Clone()\n\n\tif err := kv.Close(); err != nil {\n\t\tt.Fatalf(\"Closing the dummy kv failed: %v\", err)\n\t}\n\n\tkv, err = db.NewDiskDatabase(dbPath)\n\tif err != nil {\n\t\tt.Fatalf(\"Could not create second dummy kv: %v\", err)\n\t}\n\n\tlkr = NewLinker(kv)\n\thead, err = lkr.Head()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to retrieve head after kv reload: %v\", err)\n\t}\n\n\tif !oldHeadHash.Equal(head.TreeHash()) {\n\t\tt.Fatalf(\"HEAD hash differs before and after reload: %v <-> %v\", oldHeadHash, head.TreeHash())\n\t}\n\n\tif err := kv.Close(); err != nil {\n\t\tt.Fatalf(\"Closing the second kv failed: %v\", err)\n\t}\n}\n\nfunc TestCollideSameObjectHash(t *testing.T) {\n\tWithDummyKv(t, func(kv db.Database) {\n\t\tlkr := NewLinker(kv)\n\t\troot, err := lkr.Root()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to retrieve root: %v\", err)\n\t\t}\n\n\t\tsub, err := n.NewEmptyDirectory(lkr, root, \"sub\", \"u\", 3)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Creating empty sub dir failed: %v\", err)\n\t\t\treturn\n\t\t}\n\n\t\tif err := lkr.StageNode(sub); err != nil {\n\t\t\tt.Fatalf(\"Staging /sub failed: %v\", err)\n\t\t}\n\n\t\tfile1 := n.NewEmptyFile(sub, \"a.png\", \"u\", 4)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to create empty file1: %v\", err)\n\t\t}\n\n\t\tfile2 := n.NewEmptyFile(root, \"a.png\", \"u\", 5)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to create empty file2: %v\", err)\n\t\t}\n\n\t\tfile3 := n.NewEmptyFile(root, \"b.png\", \"u\", 6)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to create empty file3: %v\", err)\n\t\t}\n\n\t\tfile1.SetContent(lkr, h.TestDummy(t, 1))\n\t\tfile2.SetContent(lkr, h.TestDummy(t, 1))\n\t\tfile3.SetContent(lkr, h.TestDummy(t, 1))\n\n\t\tif err := sub.Add(lkr, file1); err != nil {\n\t\t\tt.Fatalf(\"Failed to add file1: %v\", err)\n\t\t}\n\t\tif err := root.Add(lkr, file2); err != nil {\n\t\t\tt.Fatalf(\"Failed to add file2: %v\", err)\n\t\t}\n\t\tif err := root.Add(lkr, file3); err != nil {\n\t\t\tt.Fatalf(\"Failed to add file3: %v\", err)\n\t\t}\n\n\t\tif err := lkr.StageNode(file1); err != nil {\n\t\t\tt.Fatalf(\"Failed to stage file1: %v\", err)\n\t\t}\n\t\tif err := lkr.StageNode(file2); err != nil {\n\t\t\tt.Fatalf(\"Failed to stage file2: %v\", err)\n\t\t}\n\t\tif err := lkr.StageNode(file3); err != nil {\n\t\t\tt.Fatalf(\"Failed to stage file3: %v\", err)\n\t\t}\n\n\t\tif file1.TreeHash().Equal(file2.TreeHash()) {\n\t\t\tt.Fatalf(\"file1 and file2 hash is equal: %v\", file1.TreeHash())\n\t\t}\n\t\tif file2.TreeHash().Equal(file3.TreeHash()) {\n\t\t\tt.Fatalf(\"file2 and file3 hash is equal: %v\", file2.TreeHash())\n\t\t}\n\n\t\t// Make sure we load the actual hashes from disk:\n\t\tlkr.MemIndexClear()\n\t\tfile1Reset, err := lkr.LookupFile(\"/sub/a.png\")\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Re-Lookup of file1 failed: %v\", err)\n\t\t}\n\t\tfile2Reset, err := lkr.LookupFile(\"/a.png\")\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Re-Lookup of file2 failed: %v\", err)\n\t\t}\n\t\tfile3Reset, err := lkr.LookupFile(\"/b.png\")\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Re-Lookup of file3 failed: %v\", err)\n\t\t}\n\n\t\tif file1Reset.TreeHash().Equal(file2Reset.TreeHash()) {\n\t\t\tt.Fatalf(\"file1Reset and file2Reset hash is equal: %v\", file1.TreeHash())\n\t\t}\n\t\tif file2Reset.TreeHash().Equal(file3Reset.TreeHash()) {\n\t\t\tt.Fatalf(\"file2Reset and file3Reset hash is equal: %v\", file2.TreeHash())\n\t\t}\n\t})\n}\n\nfunc TestHaveStagedChanges(t *testing.T) {\n\tWithDummyLinker(t, func(lkr *Linker) {\n\t\thasChanges, err := lkr.HaveStagedChanges()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"have staged changes failed before touch: %v\", err)\n\t\t}\n\t\tif hasChanges {\n\t\t\tt.Fatalf(\"HaveStagedChanges has changes before something happened\")\n\t\t}\n\n\t\tMustTouch(t, lkr, \"/x.png\", 1)\n\n\t\thasChanges, err = lkr.HaveStagedChanges()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"have staged changes failed after touch: %v\", err)\n\t\t}\n\t\tif !hasChanges {\n\t\t\tt.Fatalf(\"HaveStagedChanges has no changes after something happened\")\n\t\t}\n\n\t\tMustCommit(t, lkr, \"second\")\n\n\t\thasChanges, err = lkr.HaveStagedChanges()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"have staged changes failed after commit: %v\", err)\n\t\t}\n\t\tif hasChanges {\n\t\t\tt.Fatalf(\"HaveStagedChanges has changes after commit\")\n\t\t}\n\t})\n}\n\nfunc TestFilesByContent(t *testing.T) {\n\tWithDummyLinker(t, func(lkr *Linker) {\n\t\tfile := MustTouch(t, lkr, \"/x.png\", 1)\n\n\t\tcontents := []h.Hash{file.BackendHash()}\n\t\tresult, err := lkr.FilesByContents(contents)\n\n\t\trequire.Nil(t, err)\n\n\t\tresultFile, ok := result[file.BackendHash().B58String()]\n\t\trequire.True(t, ok)\n\t\trequire.Len(t, result, 1)\n\t\trequire.Equal(t, file, resultFile)\n\t})\n}\n\nfunc TestResolveRef(t *testing.T) {\n\tWithDummyLinker(t, func(lkr *Linker) {\n\t\tinitCmt, err := lkr.Head()\n\t\trequire.Nil(t, err)\n\n\t\tcmts := []*n.Commit{initCmt}\n\t\tfor idx := 0; idx < 10; idx++ {\n\t\t\t_, cmt := MustTouchAndCommit(t, lkr, \"/x\", byte(idx))\n\t\t\tcmts = append([]*n.Commit{cmt}, cmts...)\n\t\t}\n\n\t\t// Insert the init cmt a few times as fodder:\n\t\tcmts = append(cmts, initCmt)\n\t\tcmts = append(cmts, initCmt)\n\t\tcmts = append(cmts, initCmt)\n\n\t\tfor nUp := 0; nUp < len(cmts)+3; nUp++ {\n\t\t\trefname := \"head\"\n\t\t\tfor idx := 0; idx < nUp; idx++ {\n\t\t\t\trefname += \"^\"\n\t\t\t}\n\n\t\t\texpect := initCmt\n\t\t\tif nUp < len(cmts) {\n\t\t\t\texpect = cmts[nUp]\n\t\t\t}\n\n\t\t\tref, err := lkr.ResolveRef(refname)\n\t\t\trequire.Nil(t, err)\n\t\t\trequire.Equal(t, expect, ref)\n\t\t}\n\n\t\t_, err = lkr.ResolveRef(\"he^^ad\")\n\t\trequire.Equal(t, err, ie.ErrNoSuchRef(\"he^^ad\"))\n\t})\n}\n\ntype iterResult struct {\n\tpath, commit string\n}\n\nfunc TestIterAll(t *testing.T) {\n\tWithDummyLinker(t, func(lkr *Linker) {\n\t\tinit, err := lkr.Head()\n\t\trequire.Nil(t, err)\n\t\tc0 := init.TreeHash().B58String()\n\n\t\tx := MustTouch(t, lkr, \"/x\", 1)\n\t\tMustTouch(t, lkr, \"/y\", 1)\n\t\tfirst := MustCommit(t, lkr, \"first\")\n\t\tc1 := first.TreeHash().B58String()\n\t\tMustModify(t, lkr, x, 2)\n\n\t\tstatus, err := lkr.Status()\n\t\trequire.Nil(t, err)\n\t\tc2 := status.TreeHash().B58String()\n\n\t\tresults := []iterResult{}\n\t\trequire.Nil(t, lkr.IterAll(nil, nil, func(nd n.ModNode, cmt *n.Commit) error {\n\t\t\tresults = append(results, iterResult{nd.Path(), cmt.TreeHash().B58String()})\n\t\t\treturn nil\n\t\t}))\n\n\t\tsort.Slice(results, func(i, j int) bool {\n\t\t\t// Do not change orderings between commits:\n\t\t\tif results[i].commit != results[j].commit {\n\t\t\t\treturn false\n\t\t\t}\n\n\t\t\treturn results[i].path < results[j].path\n\t\t})\n\n\t\texpected := []iterResult{\n\t\t\t{\"/\", c2},\n\t\t\t{\"/x\", c2},\n\t\t\t{\"/y\", c2},\n\t\t\t{\"/\", c1},\n\t\t\t{\"/x\", c1},\n\t\t\t{\"/\", c0},\n\t\t}\n\n\t\tfor idx, result := range results {\n\t\t\trequire.Equal(t, result.path, expected[idx].path)\n\t\t\trequire.Equal(t, result.commit, expected[idx].commit)\n\t\t}\n\t})\n}\n\nfunc TestAtomic(t *testing.T) {\n\tWithDummyLinker(t, func(lkr *Linker) {\n\t\terr := lkr.Atomic(func() (bool, error) {\n\t\t\tMustTouch(t, lkr, \"/x\", 1)\n\t\t\treturn false, nil\n\t\t})\n\n\t\trequire.Nil(t, err)\n\n\t\terr = lkr.Atomic(func() (bool, error) {\n\t\t\tMustTouch(t, lkr, \"/y\", 1)\n\t\t\treturn true, errors.New(\"artificial error\")\n\t\t})\n\n\t\trequire.NotNil(t, err)\n\n\t\terr = lkr.Atomic(func() (bool, error) {\n\t\t\tMustTouch(t, lkr, \"/z\", 1)\n\t\t\tpanic(\"woah\")\n\t\t})\n\n\t\trequire.NotNil(t, err)\n\n\t\tx, err := lkr.LookupFile(\"/x\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, x.Path(), \"/x\")\n\n\t\t_, err = lkr.LookupFile(\"/y\")\n\t\trequire.NotNil(t, err)\n\t\trequire.True(t, ie.IsNoSuchFileError(err))\n\n\t\t_, err = lkr.LookupFile(\"/z\")\n\t\trequire.NotNil(t, err)\n\t\trequire.True(t, ie.IsNoSuchFileError(err))\n\t})\n\n}\n\nfunc TestCommitByIndex(t *testing.T) {\n\t// Note: WithReloadingLinker creates an init commit.\n\tWithDummyLinker(t, func(lkr *Linker) {\n\t\thead, err := lkr.Head()\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, head.Index(), int64(0))\n\n\t\tstatus, err := lkr.Status()\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, int64(1), status.Index())\n\n\t\t// Must modify something to commit:\n\t\tMustTouch(t, lkr, \"/x\", 1)\n\n\t\trequire.Nil(t, lkr.MakeCommit(\"me\", \"is mario\"))\n\t\tnewHead, err := lkr.Head()\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, int64(1), newHead.Index())\n\n\t\tstatus, err = lkr.Status()\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, int64(2), status.Index())\n\n\t\t// Lookup the just created commits:\n\n\t\t// Pre-existing init commit:\n\t\tc1, err := lkr.CommitByIndex(0)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, \"init\", c1.Message())\n\n\t\t// Our commit:\n\t\tc2, err := lkr.CommitByIndex(1)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, \"is mario\", c2.Message())\n\n\t\t// Same as the status commit:\n\t\tc3, err := lkr.CommitByIndex(2)\n\t\trequire.Nil(t, err)\n\t\trequire.NotNil(t, c3)\n\t\trequire.Equal(t, status.TreeHash(), c3.TreeHash())\n\n\t\t// Not existing:\n\t\tc4, err := lkr.CommitByIndex(3)\n\t\trequire.True(t, ie.IsErrNoSuchCommitIndex(err))\n\t\trequire.Nil(t, c4)\n\t})\n}\n\nfunc TestLookupNodeAt(t *testing.T) {\n\tWithDummyLinker(t, func(lkr *Linker) {\n\t\tfmt.Println(\"start\")\n\t\tfor idx := byte(0); idx < 10; idx++ {\n\t\t\tMustTouchAndCommit(t, lkr, \"/x\", idx)\n\t\t}\n\t\tfmt.Println(\"done\")\n\n\t\tfor idx := 0; idx < 10; idx++ {\n\t\t\t// commit index of 0 is init, so + 1\n\t\t\tcmt, err := lkr.CommitByIndex(int64(idx + 1))\n\t\t\trequire.Nil(t, err)\n\n\t\t\tnd, err := lkr.LookupNodeAt(cmt, \"/x\")\n\t\t\trequire.Nil(t, err)\n\t\t\trequire.Equal(t, nd.ContentHash(), h.TestDummy(t, byte(idx)))\n\t\t}\n\n\t\t// Init should not exist:\n\t\tinit, err := lkr.CommitByIndex(0)\n\t\trequire.Nil(t, err)\n\n\t\tnd, err := lkr.LookupNodeAt(init, \"/x\")\n\t\trequire.Nil(t, nd)\n\t\trequire.True(t, ie.IsNoSuchFileError(err))\n\n\t\t// Stage should have the last change:\n\t\tstage, err := lkr.CommitByIndex(11)\n\t\trequire.Nil(t, err)\n\n\t\tstageNd, err := lkr.LookupNodeAt(stage, \"/x\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, stageNd.ContentHash(), h.TestDummy(t, 9))\n\n\t\t// quick check to see if the next commit is really empty\n\t\t// (tests only the test setup)\n\t\tlast, err := lkr.CommitByIndex(12)\n\t\trequire.True(t, ie.IsErrNoSuchCommitIndex(err))\n\t\trequire.Nil(t, last)\n\t})\n}\n" }, { "path": "catfs/core/testing.go", "content": "package core\n\nimport (\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"path\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/sahib/brig/catfs/db\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// WithDummyKv creates a testing key value store and passes it to `fn`.\nfunc WithDummyKv(t *testing.T, fn func(kv db.Database)) {\n\tdbPath, err := ioutil.TempDir(\"\", \"brig-test\")\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to create temp dir: %v\", err)\n\t}\n\n\tdefer os.RemoveAll(dbPath)\n\n\tkv, err := db.NewDiskDatabase(dbPath)\n\tif err != nil {\n\t\tt.Fatalf(\"Could not create dummy kv for tests: %v\", err)\n\t}\n\n\tfn(kv)\n\n\tif err := kv.Close(); err != nil {\n\t\tt.Fatalf(\"Closing the dummy kv failed: %v\", err)\n\t}\n}\n\n// WithDummyLinker creates a testing linker and passes it to `fn`.\nfunc WithDummyLinker(t *testing.T, fn func(lkr *Linker)) {\n\tWithDummyKv(t, func(kv db.Database) {\n\t\tlkr := NewLinker(kv)\n\t\trequire.Nil(t, lkr.SetOwner(\"alice\"))\n\t\tMustCommit(t, lkr, \"init\")\n\n\t\tfn(lkr)\n\t})\n}\n\n// WithReloadingLinker creates a testing linker and passes it to `fn1`.\n// It then closes the linker and lets it load a second time and passes it to `fn2`.\n// This is useful to test persistency issues.\nfunc WithReloadingLinker(t *testing.T, fn1 func(lkr *Linker), fn2 func(lkr *Linker)) {\n\tWithDummyKv(t, func(kv db.Database) {\n\t\tlkr1 := NewLinker(kv)\n\t\trequire.Nil(t, lkr1.SetOwner(\"alice\"))\n\t\tMustCommit(t, lkr1, \"init\")\n\n\t\tfn1(lkr1)\n\n\t\tlkr2 := NewLinker(kv)\n\t\tfn2(lkr2)\n\t})\n}\n\n// WithLinkerPair creates two linkers, useful for testing syncing.\nfunc WithLinkerPair(t *testing.T, fn func(lkrSrc, lkrDst *Linker)) {\n\tWithDummyLinker(t, func(lkrSrc *Linker) {\n\t\tWithDummyLinker(t, func(lkrDst *Linker) {\n\t\t\trequire.Nil(t, lkrSrc.SetOwner(\"src\"))\n\t\t\trequire.Nil(t, lkrDst.SetOwner(\"dst\"))\n\t\t\tfn(lkrSrc, lkrDst)\n\t\t})\n\t})\n}\n\n// AssertDir asserts the existence of a directory.\nfunc AssertDir(t *testing.T, lkr *Linker, path string, shouldExist bool) {\n\tdir, err := lkr.LookupDirectory(path)\n\tif shouldExist {\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"exist-check: Directory lookup failed for %s: %v\", path, err)\n\t\t}\n\n\t\tif dir == nil || dir.Path() != path {\n\t\t\tt.Fatalf(\"exist-check: directory does not exist: %s -> %v\", path, dir)\n\t\t}\n\t} else {\n\t\tif dir != nil {\n\t\t\tt.Fatalf(\"exist-check: Dir exists, but should not: %v\", path)\n\t\t}\n\t}\n}\n\n// MustMkdir creates a directory or fails on `t`.\nfunc MustMkdir(t *testing.T, lkr *Linker, repoPath string) *n.Directory {\n\tdir, err := Mkdir(lkr, repoPath, true)\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to create directories %s: %v\", repoPath, err)\n\t}\n\n\treturn dir\n}\n\n// MustTouch creates a new node at `touchPath` and sets its content hash\n// to a hash derived from `seed`.\nfunc MustTouch(t *testing.T, lkr *Linker, touchPath string, seed byte) *n.File {\n\tdirname := path.Dir(touchPath)\n\tparent, err := lkr.LookupDirectory(dirname)\n\tif err != nil {\n\t\tt.Fatalf(\"touch: Failed to lookup: %s\", dirname)\n\t}\n\n\tbasePath := path.Base(touchPath)\n\tfile := n.NewEmptyFile(parent, basePath, lkr.owner, lkr.NextInode())\n\n\tfile.SetBackend(lkr, h.TestDummy(t, seed))\n\tfile.SetContent(lkr, h.TestDummy(t, seed))\n\tfile.SetKey(make([]byte, 32))\n\n\tchild, err := parent.Child(lkr, basePath)\n\tif err != nil {\n\t\tt.Fatalf(\"touch: Failed to lookup child: %v %v\", touchPath, err)\n\t}\n\n\tif child != nil {\n\t\tif err := parent.RemoveChild(lkr, child); err != nil {\n\t\t\tt.Fatalf(\"touch: failed to remove previous node: %v\", err)\n\t\t}\n\t}\n\n\tif err := parent.Add(lkr, file); err != nil {\n\t\tt.Fatalf(\"touch: Adding %s to root failed: %v\", touchPath, err)\n\t}\n\n\tif err := lkr.StageNode(file); err != nil {\n\t\tt.Fatalf(\"touch: Staging %s failed: %v\", touchPath, err)\n\t}\n\n\treturn file\n}\n\n// MustMove moves the node `nd` to `destPath` or fails `t`.\nfunc MustMove(t *testing.T, lkr *Linker, nd n.ModNode, destPath string) n.ModNode {\n\tif err := Move(lkr, nd, destPath); err != nil {\n\t\tt.Fatalf(\"move of %s to %s failed: %v\", nd.Path(), destPath, err)\n\t}\n\n\tnewNd, err := lkr.LookupModNode(destPath)\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to lookup dest path `%s` of new node: %v\", destPath, err)\n\t}\n\n\treturn newNd\n}\n\n// MustRemove removes the node `nd` or fails.\nfunc MustRemove(t *testing.T, lkr *Linker, nd n.ModNode) n.ModNode {\n\tif _, _, err := Remove(lkr, nd, true, false); err != nil {\n\t\tt.Fatalf(\"Failed to remove %s: %v\", nd.Path(), err)\n\t}\n\n\tnewNd, err := lkr.LookupModNode(nd.Path())\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to lookup dest path `%s` of deleted node: %v\", nd.Path(), err)\n\t}\n\n\treturn newNd\n}\n\n// MustCommit commits the current state with `msg`.\nfunc MustCommit(t *testing.T, lkr *Linker, msg string) *n.Commit {\n\tif err := lkr.MakeCommit(n.AuthorOfStage, msg); err != nil {\n\t\tt.Fatalf(\"Failed to make commit with msg %s: %v\", msg, err)\n\t}\n\n\thead, err := lkr.Head()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to retrieve head after commit: %v\", err)\n\t}\n\n\treturn head\n}\n\n// MustCommitIfPossible with is like MustCommit, but allows empty changesets.\nfunc MustCommitIfPossible(t *testing.T, lkr *Linker, msg string) *n.Commit {\n\thaveChanges, err := lkr.HaveStagedChanges()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to check for changes: %v\", err)\n\t}\n\n\tif haveChanges {\n\t\treturn MustCommit(t, lkr, msg)\n\t}\n\n\treturn nil\n}\n\n// MustTouchAndCommit is a combined MustTouch and MustCommit.\nfunc MustTouchAndCommit(t *testing.T, lkr *Linker, path string, seed byte) (*n.File, *n.Commit) {\n\tfile, err := Stage(\n\t\tlkr,\n\t\tpath,\n\t\th.TestDummy(t, seed),\n\t\th.TestDummy(t, seed),\n\t\tuint64(seed),\n\t\t-1,\n\t\tnil,\n\t\ttime.Now(),\n\t\tfalse,\n\t)\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to stage %s at %d: %v\", path, seed, err)\n\t}\n\n\treturn file, MustCommit(t, lkr, fmt.Sprintf(\"cmt %d\", seed))\n}\n\n// MustModify changes the content of an existing node.\nfunc MustModify(t *testing.T, lkr *Linker, file *n.File, seed int) {\n\tparent, err := lkr.LookupDirectory(path.Dir(file.Path()))\n\t// root, err := lkr.Root()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to get root: %v\", err)\n\t}\n\n\tif err := parent.RemoveChild(lkr, file); err != nil && !ie.IsNoSuchFileError(err) {\n\t\tt.Fatalf(\"Unable to remove %s from /: %v\", file.Path(), err)\n\t}\n\n\tfile.SetSize(uint64(seed))\n\tfile.SetBackend(lkr, h.TestDummy(t, byte(seed)))\n\tfile.SetContent(lkr, h.TestDummy(t, byte(seed)))\n\n\tif err := parent.Add(lkr, file); err != nil {\n\t\tt.Fatalf(\"Unable to add %s to /: %v\", file.Path(), err)\n\t}\n\n\tif err := lkr.StageNode(file); err != nil {\n\t\tt.Fatalf(\"Failed to stage %s for second: %v\", file.Path(), err)\n\t}\n}\n\n// MustLookupDirectory loads an existing dir or fails.\nfunc MustLookupDirectory(t *testing.T, lkr *Linker, path string) *n.Directory {\n\tdir, err := lkr.LookupDirectory(path)\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to lookup directory %v: %v\", path, err)\n\t}\n\n\treturn dir\n}\n" }, { "path": "catfs/db/database.go", "content": "package db\n\nimport (\n\t\"errors\"\n\t\"io\"\n)\n\nvar (\n\t// ErrNoSuchKey is returned when Get() was passed a non-existent key\n\tErrNoSuchKey = errors.New(\"This key does not exist\")\n)\n\n// Batch is an API object used to model a transaction.\ntype Batch interface {\n\t// Put sets `val` at `key`.\n\tPut(val []byte, key ...string)\n\n\t// Clear all contents below and including `key`.\n\tClear(key ...string) error\n\n\t// Erase a key from the database.\n\tErase(key ...string)\n\n\t// Flush the batch to the database.\n\t// Only now, all changes will be written to disk.\n\tFlush() error\n\n\t// Rollback will forget all changes without executing them.\n\tRollback()\n\n\t// HaveWrites returns true when the batch contains something\n\t// we can write to the disk on Flush().\n\tHaveWrites() bool\n}\n\n// Database is a key/value store that offers different buckets\n// for storage. Keys are strings, values are arbitrary untyped data.\ntype Database interface {\n\t// Get retrievies the key `key` out of bucket.\n\t// If no such key exists, it will return (nil, ErrNoSuchKey)\n\t// If a badge is currently open, Get() shall still return the\n\t// most current value currently set by the last Put() call\n\t// to `key`.\n\tGet(key ...string) ([]byte, error)\n\n\t// Keys iterates over all keys in the database If the error is returned by\n\t// `fn` the iteration stops and the error value is returned.\n\t// The keys are returned in lexical ordering.\n\tKeys(prefix ...string) ([][]string, error)\n\n\t// Batch returns a new Batch object, that will allow modifications\n\t// of the state. Batch() can be called recursive: The changes will\n\t// only be flushed to disk if batch.Flush() was called equal times\n\t// to the number Batch() was called.\n\tBatch() Batch\n\n\t// Export backups all database content to `w` in\n\t// an implemenation specific format that can be read by Import.\n\tExport(w io.Writer) error\n\n\t// Import reads a previously exported db dump by Export from `r`.\n\t// Existing keys might be overwritten if the dump also contains them.\n\tImport(r io.Reader) error\n\n\t// Close closes the database. Since I/O may happen, an error is returned.\n\tClose() error\n\n\t// Glob finds all existing keys in the store, starting with prefix.\n\tGlob(prefix []string) ([][]string, error)\n}\n\n// CopyKey is a helper method to copy a bunch of keys in `src` to `dst`.\nfunc CopyKey(db Database, src, dst []string) error {\n\tdata, err := db.Get(src...)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tbatch := db.Batch()\n\tbatch.Put(data, dst...)\n\treturn batch.Flush()\n}\n" }, { "path": "catfs/db/database_badger.go", "content": "package db\n\nimport (\n\t\"io\"\n\t\"strings\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"time\"\n\n\tbadger \"github.com/dgraph-io/badger/v3\"\n\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// BadgerDatabase is a database implementation based on BadgerDB\ntype BadgerDatabase struct {\n\tmu sync.Mutex\n\tisStopped int64\n\tdb *badger.DB\n\ttxn *badger.Txn\n\trefCount int\n\thaveWrites bool\n\twriteTimes []time.Time\n\tgcTicker *time.Ticker\n}\n\n// NewBadgerDatabase creates a new badger database.\nfunc NewBadgerDatabase(path string) (*BadgerDatabase, error) {\n\topts := badger.DefaultOptions(path).\n\t\tWithValueLogFileSize(10 * 1024 * 1024). //default is 2GB we should not need 2GB\n\t\tWithMemTableSize(10 * 1024 * 1024). //default is 64MB\n\t\tWithNumVersionsToKeep(1). // it is default but it's better to force it\n\t\tWithCompactL0OnClose(true).\n\t\tWithSyncWrites(false).\n\t\tWithLogger(nil)\n\n\tdb, err := badger.Open(opts)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tgcTicker := time.NewTicker(5 * time.Minute)\n\n\tbdb := &BadgerDatabase{\n\t\tdb: db,\n\t\tgcTicker: gcTicker,\n\t}\n\n\tgo func() {\n\t\tfor range gcTicker.C {\n\t\t\tif atomic.LoadInt64(&bdb.isStopped) > 0 {\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\terr := bdb.runGC()\n\t\t\tif err != nil {\n\t\t\t\tlog.WithError(err).Error(\"badger GC failed\")\n\t\t\t}\n\t\t}\n\t}()\n\n\treturn bdb, nil\n}\n\nfunc (bdb *BadgerDatabase) runGC() error {\n\topts := bdb.db.Opts()\n\tbdb.mu.Lock()\n\tdefer bdb.mu.Unlock()\n\tlog.Debugf(\"Performing GC for badger DB in %s\", opts.Dir)\n\ttStart := time.Now()\n\tdefer func() {\n\t\tlog.Debugf(\"GC collection on %s took %v\", opts.Dir, time.Now().Sub(tStart))\n\t}()\n\t// we will go through array of write times to see if it is time to run GC\n\tvar gcStatsUpdateDelay = 5 * time.Minute\n\tvar deadlineMet = false\n\tn := 0\n\tfor _, t := range bdb.writeTimes {\n\t\tif time.Now().Before(t.Add(gcStatsUpdateDelay)) {\n\t\t\tbdb.writeTimes[n] = t\n\t\t\tn++\n\t\t} else {\n\t\t\tdeadlineMet = true\n\t\t}\n\t}\n\tbdb.writeTimes = bdb.writeTimes[:n]\n\tif !deadlineMet {\n\t\tlog.Debugf(\"DB in %s has no new stats for GC\", opts.Dir)\n\t\treturn nil\n\t}\n\t// In large DB, GC will happen automatically, because compaction will find garbage\n\t// but we are to small and compactors do not run (150 MB is small).\n\t// So we need to run Flatten\n\tbdb.db.Flatten(5)\n\t// Very likely Flatten will not do much because the hard coded priority is too small.\n\t// At this point, we hope that there is something for GC\n\tvar errGC error\n\tvar success = false\n\tfor errGC == nil {\n\t\t// cleans DB online and it is safe to rerun on success\n\t\terrGC = bdb.db.RunValueLogGC(0.5)\n\t\tif errGC == nil && !success {\n\t\t\tsuccess = true\n\t\t}\n\t}\n\tif success {\n\t\tlog.Debugf(\"Cleaned some garbage for DB in %s\", opts.Dir)\n\t\treturn nil\n\t}\n\t// Now we have a dilemma: we could trust badger GC mechanism and stop here.\n\t// But unfortunately for our typical size (even as high as 150 MB)\n\t// compaction, even with Flatten(), does not kick in.\n\t// The only way to truly force compaction (to update stats for GC) is to close DB\n\t// see Note in https://github.com/dgraph-io/badger/issues/767#issuecomment-485713746\n\t// After Close() the GC on a next run will have updated statistic\n\t// Actually even Close() does not guaranteed success, it requires more than a minute\n\t// to update stats after DB was modified. But eventually GC stats will be ready.\n\tif bdb.txn != nil {\n\t\t// someone still using DB, we will try to Close/Open next time\n\t\treturn nil\n\t}\n\tvar err error\n\tfor retries := 0; retries < 10; retries++ {\n\t\terr = bdb.db.Close()\n\t\tif err == nil {\n\t\t\tcontinue\n\t\t}\n\t\tlog.Errorf(\"Could not close DB in %s\", opts.Dir)\n\t\ttime.Sleep(1 * time.Second)\n\t}\n\tif err != nil {\n\t\tlog.Fatalf(\"Could not close DB in %s\", opts.Dir)\n\t\treturn err\n\t}\n\tfor retries := 0; retries < 10; retries++ {\n\t\tbdb.db, err = badger.Open(opts)\n\t\tif err == nil {\n\t\t\treturn nil\n\t\t}\n\t\tlog.Errorf(\"Could not reopen DB in %s\", opts.Dir)\n\t\ttime.Sleep(1 * time.Second)\n\t}\n\tlog.Fatalf(\"Could not reopen DB in %s\", opts.Dir)\n\treturn err\n}\n\nfunc (db *BadgerDatabase) view(fn func(txn *badger.Txn) error) error {\n\t// If we have an open transaction, retrieve the values from there.\n\t// Otherwise we would not be able to retrieve in-memory values.\n\tif db.txn != nil {\n\t\treturn fn(db.txn)\n\t}\n\n\t// If no transaction is running (no Batch()-call), use a fresh view txn.\n\treturn db.db.View(fn)\n}\n\n// Get is the badger implementation of Database.Get.\nfunc (db *BadgerDatabase) Get(key ...string) ([]byte, error) {\n\tdb.mu.Lock()\n\tdefer db.mu.Unlock()\n\n\tdata := []byte{}\n\terr := db.view(func(txn *badger.Txn) error {\n\t\tif db.txn != nil {\n\t\t\ttxn = db.txn\n\t\t}\n\n\t\tkeyPath := strings.Join(key, \".\")\n\t\titem, err := txn.Get([]byte(keyPath))\n\t\tif err == badger.ErrKeyNotFound {\n\t\t\treturn ErrNoSuchKey\n\t\t}\n\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tdata, err = item.ValueCopy(nil)\n\t\treturn err\n\t})\n\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn data, nil\n}\n\n// Keys is the badger implementation of Database.Keys.\nfunc (db *BadgerDatabase) Keys(prefix ...string) ([][]string, error) {\n\tdb.mu.Lock()\n\tdefer db.mu.Unlock()\n\n\tkeys := [][]string{}\n\treturn keys, db.view(func(txn *badger.Txn) error {\n\t\titer := txn.NewIterator(badger.IteratorOptions{})\n\t\tdefer iter.Close()\n\n\t\tfor iter.Rewind(); iter.Valid(); iter.Next() {\n\t\t\titem := iter.Item()\n\n\t\t\tfullKey := string(item.Key())\n\t\t\tsplitKey := strings.Split(fullKey, \".\")\n\n\t\t\thasPrefix := len(prefix) <= len(splitKey)\n\t\t\tfor i := 0; hasPrefix && i < len(prefix) && i < len(splitKey); i++ {\n\t\t\t\tif prefix[i] != splitKey[i] {\n\t\t\t\t\thasPrefix = false\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif hasPrefix {\n\t\t\t\tkeys = append(keys, strings.Split(fullKey, \".\"))\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t})\n}\n\n// Export is the badger implementation of Database.Export.\nfunc (db *BadgerDatabase) Export(w io.Writer) error {\n\tdb.mu.Lock()\n\tdefer db.mu.Unlock()\n\n\t_, err := db.db.Backup(w, 0)\n\treturn err\n}\n\n// Import is the badger implementation of Database.Import.\nfunc (db *BadgerDatabase) Import(r io.Reader) error {\n\tdb.mu.Lock()\n\tdefer db.mu.Unlock()\n\n\treturn db.db.Load(r, 1)\n}\n\n// Glob is the badger implementation of Database.Glob\nfunc (db *BadgerDatabase) Glob(prefix []string) ([][]string, error) {\n\tdb.mu.Lock()\n\tdefer db.mu.Unlock()\n\n\tfullPrefix := strings.Join(prefix, \".\")\n\n\tresults := [][]string{}\n\terr := db.view(func(txn *badger.Txn) error {\n\t\titer := txn.NewIterator(badger.IteratorOptions{})\n\t\tdefer iter.Close()\n\n\t\tfor iter.Seek([]byte(fullPrefix)); iter.Valid(); iter.Next() {\n\t\t\tfullKey := string(iter.Item().Key())\n\t\t\tif !strings.HasPrefix(fullKey, fullPrefix) {\n\t\t\t\tbreak\n\t\t\t}\n\n\t\t\t// Don't do recursive globbing:\n\t\t\tleftOver := fullKey[len(fullPrefix):]\n\t\t\tif !strings.Contains(leftOver, \".\") {\n\t\t\t\tresults = append(results, strings.Split(fullKey, \".\"))\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t})\n\n\treturn results, err\n}\n\n// Batch is the badger implementation of Database.Batch\nfunc (db *BadgerDatabase) Batch() Batch {\n\tdb.mu.Lock()\n\tdefer db.mu.Unlock()\n\n\treturn db.batch()\n}\n\nfunc (db *BadgerDatabase) batch() Batch {\n\tif db.txn == nil {\n\t\tdb.txn = db.db.NewTransaction(true)\n\t}\n\n\tdb.refCount++\n\treturn db\n}\n\n// Put is the badger implementation of Database.Put\nfunc (db *BadgerDatabase) Put(val []byte, key ...string) {\n\tdb.mu.Lock()\n\tdefer db.mu.Unlock()\n\n\tdb.haveWrites = true\n\n\tfullKey := []byte(strings.Join(key, \".\"))\n\n\terr := db.withRetry(func() error {\n\t\treturn db.txn.Set(fullKey, val)\n\t})\n\n\tif err != nil {\n\t\tlog.Warningf(\"badger: failed to set key %s: %v\", fullKey, err)\n\t}\n}\n\nfunc (db *BadgerDatabase) withRetry(fn func() error) error {\n\tif err := fn(); err != badger.ErrTxnTooBig {\n\t\t// This also returns nil.\n\t\treturn err\n\t}\n\n\t// Commit previous (almost too big) transaction:\n\tif err := db.txn.Commit(); err != nil {\n\t\t// Something seems pretty wrong.\n\t\treturn err\n\t}\n\n\tdb.txn = db.db.NewTransaction(true)\n\n\t// If this fails again, we're out of luck.\n\treturn fn()\n}\n\n// Clear is the badger implementation of Database.Clear\nfunc (db *BadgerDatabase) Clear(key ...string) error {\n\tdb.mu.Lock()\n\tdefer db.mu.Unlock()\n\n\tdb.haveWrites = true\n\n\titer := db.txn.NewIterator(badger.IteratorOptions{})\n\tprefix := strings.Join(key, \".\")\n\n\tkeys := [][]byte{}\n\tfor iter.Rewind(); iter.Valid(); iter.Next() {\n\t\titem := iter.Item()\n\n\t\tkey := make([]byte, len(item.Key()))\n\t\tcopy(key, item.Key())\n\t\tkeys = append(keys, key)\n\t}\n\n\t// This has to happen here, since withRetry might call\n\t// txn.Discard() which will complain about open iterators.\n\t// (I previously used a defer which executed too late)\n\titer.Close()\n\n\tfor _, key := range keys {\n\t\tif !strings.HasPrefix(string(key), prefix) {\n\t\t\tcontinue\n\t\t}\n\n\t\terr := db.withRetry(func() error {\n\t\t\treturn db.txn.Delete(key)\n\t\t})\n\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// Erase is the badger implementation of Database.Erase\nfunc (db *BadgerDatabase) Erase(key ...string) {\n\tdb.mu.Lock()\n\tdefer db.mu.Unlock()\n\n\tdb.haveWrites = true\n\n\tfullKey := []byte(strings.Join(key, \".\"))\n\terr := db.withRetry(func() error {\n\t\treturn db.txn.Delete(fullKey)\n\t})\n\n\tif err != nil {\n\t\tlog.Warningf(\"badger: failed to del key %s: %v\", fullKey, err)\n\t}\n}\n\n// Flush is the badger implementation of Database.Flush\nfunc (db *BadgerDatabase) Flush() error {\n\tdb.mu.Lock()\n\tdefer db.mu.Unlock()\n\n\tdb.refCount--\n\tif db.refCount > 0 {\n\t\treturn nil\n\t}\n\n\tif db.refCount < 0 {\n\t\tlog.Errorf(\"negative batch ref count: %d\", db.refCount)\n\t\treturn nil\n\t}\n\n\tdefer db.txn.Discard()\n\tif err := db.txn.Commit(); err != nil {\n\t\treturn err\n\t}\n\n\tdb.txn = nil\n\tif db.haveWrites {\n\t\tdb.writeTimes = append(db.writeTimes, time.Now())\n\t}\n\tdb.haveWrites = false\n\treturn nil\n}\n\n// Rollback is the badger implementation of Database.Rollback\nfunc (db *BadgerDatabase) Rollback() {\n\tdb.mu.Lock()\n\tdefer db.mu.Unlock()\n\n\tdb.refCount--\n\tif db.refCount > 0 {\n\t\treturn\n\t}\n\n\tif db.refCount < 0 {\n\t\tlog.Errorf(\"negative batch ref count: %d\", db.refCount)\n\t\treturn\n\t}\n\n\tdb.txn.Discard()\n\tdb.txn = nil\n\tdb.haveWrites = false\n\tdb.refCount = 0\n}\n\n// HaveWrites is the badger implementation of Database.HaveWrites\nfunc (db *BadgerDatabase) HaveWrites() bool {\n\tdb.mu.Lock()\n\tdefer db.mu.Unlock()\n\n\treturn db.haveWrites\n}\n\n// Close is the badger implementation of Database.Close\nfunc (db *BadgerDatabase) Close() error {\n\tdb.mu.Lock()\n\tdefer db.mu.Unlock()\n\n\tdb.gcTicker.Stop()\n\tatomic.StoreInt64(&db.isStopped, 1)\n\n\t// With an open transaction it would deadlock:\n\tif db.txn != nil {\n\t\tdb.txn.Discard()\n\t\tdb.txn = nil\n\t\tdb.haveWrites = false\n\t}\n\n\tif db.db != nil {\n\t\toldDb := db.db\n\t\tdb.db = nil\n\t\tif err := oldDb.Close(); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n" }, { "path": "catfs/db/database_disk.go", "content": "package db\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"path\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/sahib/brig/util\"\n)\n\nconst (\n\tdebug = false\n)\n\n// DiskDatabase is a database that simply uses the filesystem as storage.\n// Each bucket is one directory. Leaf keys are simple files.\n// The exported form of the database is simply a gzipped .tar of the directory.\n//\n// Note that this database backends was written for easy debugging.\n// It is currently by no means optimized for fast reads and writes and\n// could be probably made a lot faster if we ever need that.\ntype DiskDatabase struct {\n\tbasePath string\n\tcache map[string][]byte\n\tops []func() error\n\trefs int64\n\tdeletes map[string]struct{}\n}\n\n// NewDiskDatabase creates a new database at `basePath`.\nfunc NewDiskDatabase(basePath string) (*DiskDatabase, error) {\n\treturn &DiskDatabase{\n\t\tbasePath: basePath,\n\t\tcache: make(map[string][]byte),\n\t\tdeletes: make(map[string]struct{}),\n\t}, nil\n}\n\nfunc fixDirectoryKeys(key []string) string {\n\tif len(key) == 0 {\n\t\treturn \"\"\n\t}\n\n\t// Filter potential \"..\" elements that could be used\n\t// to break out of the database store and read whatever files\n\t// outside of it due to bad intentions.\n\tkeyCopy := key[:0]\n\tfor _, val := range key {\n\t\tif val != \"..\" {\n\t\t\tkeyCopy = append(keyCopy, val)\n\t\t} else {\n\t\t\tkeyCopy = append(keyCopy, \"DOTDOT\")\n\t\t}\n\t}\n\n\tkey = keyCopy\n\n\tswitch lastPart := key[len(key)-1]; {\n\tcase lastPart == \"DOT\":\n\t\treturn filepath.Join(key[:len(key)-1]...) + \"/__NO_DOT__\"\n\tcase lastPart == \".\":\n\t\treturn filepath.Join(key[:len(key)-1]...) + \"/DOT\"\n\tcase strings.HasSuffix(lastPart, \"/.\"):\n\t\treturn filepath.Join(key[:len(key)-1]...) + strings.TrimRight(lastPart, \".\") + \"/DOT\"\n\tdefault:\n\t\treturn filepath.Join(key...)\n\t}\n}\n\nfunc reverseDirectoryKeys(key string) []string {\n\tparts := strings.Split(key, string(filepath.Separator))\n\tif len(parts) > 0 && parts[0] == \"\" {\n\t\tparts = parts[1:]\n\t}\n\n\tswitch parts[len(parts)-1] {\n\tcase \"DOT\":\n\t\tparts[len(parts)-1] = \".\"\n\tcase \"__NO_DOT__\":\n\t\tparts[len(parts)-1] = \"DOT\"\n\t}\n\n\treturn parts\n}\n\n// Flush is the disk implementation of Database.Flush\nfunc (db *DiskDatabase) Flush() error {\n\tdb.refs--\n\tif db.refs < 0 {\n\t\tdb.refs = 0\n\t}\n\n\tif db.refs > 0 {\n\t\treturn nil\n\t}\n\n\tif debug {\n\t\tfmt.Println(\"FLUSH\")\n\t}\n\n\t// Clear the cache first, if any of the next step fail,\n\t// we have at least the current state.\n\tdb.cache = make(map[string][]byte)\n\tdb.deletes = make(map[string]struct{})\n\n\t// Make sure that db.ops is nil, even if Flush failed.\n\tops := db.ops\n\tdb.ops = nil\n\n\t// Currently no revertible operations are implemented. If something goes\n\t// wrong on the filesystem, chances are high that we're not able to revert\n\t// previous ops anyways.\n\tfor _, op := range ops {\n\t\tif err := op(); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// Rollback is the disk implementation of Database.Rollback\nfunc (db *DiskDatabase) Rollback() {\n\tif debug {\n\t\tfmt.Println(\"ROLLBACK\")\n\t}\n\n\tdb.refs = 0\n\tdb.ops = nil\n\tdb.cache = make(map[string][]byte)\n\tdb.deletes = make(map[string]struct{})\n}\n\n// Get a single value from `bucket` by `key`.\nfunc (db *DiskDatabase) Get(key ...string) ([]byte, error) {\n\tif debug {\n\t\tfmt.Println(\"GET\", key)\n\t}\n\n\tfullKey := path.Join(key...)\n\n\t// if it's a key that was already deleted in a transaction,\n\t// we should acknowledge it as deleted.\n\tif _, ok := db.deletes[fullKey]; ok {\n\t\treturn nil, ErrNoSuchKey\n\t}\n\n\tdata, ok := db.cache[fullKey]\n\tif ok {\n\t\treturn data, nil\n\t}\n\n\t// We have to go to the disk to find the right key:\n\tfilePath := filepath.Join(db.basePath, fixDirectoryKeys(key))\n\tdata, err := ioutil.ReadFile(filePath) // #nosec\n\n\tif os.IsNotExist(err) {\n\t\treturn nil, ErrNoSuchKey\n\t}\n\n\treturn data, err\n}\n\n// Batch is the disk implementation of Database.Batch\nfunc (db *DiskDatabase) Batch() Batch {\n\tdb.refs++\n\treturn db\n}\n\nfunc removeNonDirs(path string) error {\n\tif path == \"/\" || path == \"\" {\n\t\treturn nil\n\t}\n\n\tinfo, err := os.Stat(path)\n\tif err != nil && !os.IsNotExist(err) {\n\t\treturn err\n\t}\n\n\tif info != nil && !info.IsDir() {\n\t\treturn os.Remove(path)\n\t}\n\n\treturn removeNonDirs(filepath.Dir(path))\n}\n\n// Put stores a new `val` under `key` at `bucket`.\n// Implementation detail: `key` may contain slashes (/). If used, those keys\n// will result in a nested directory structure.\nfunc (db *DiskDatabase) Put(val []byte, key ...string) {\n\tif debug {\n\t\tfmt.Println(\"SET\", key)\n\t}\n\n\tdb.ops = append(db.ops, func() error {\n\t\tfilePath := filepath.Join(db.basePath, fixDirectoryKeys(key))\n\n\t\t// If any of the parent are non-directories,\n\t\t// we need to remove them, since more nesting is requested.\n\t\t// (e.g. set /a/b/c/d over /a/b/c, where c is a file)\n\t\tparentDir := filepath.Dir(filePath)\n\t\tif err := removeNonDirs(parentDir); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif err := os.MkdirAll(parentDir, 0700); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// It is allowed to set a key over an existing one.\n\t\t// i.e. set \"a/b\" over \"a/b/c\". This requires us to potentially\n\t\t// delete nested directories (c).\n\t\tinfo, err := os.Stat(filePath)\n\t\tif err != nil && !os.IsNotExist(err) {\n\t\t\treturn err\n\t\t}\n\n\t\tif info != nil && info.IsDir() {\n\t\t\tif err := os.RemoveAll(filePath); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn ioutil.WriteFile(filePath, val, 0600)\n\t})\n\n\tfullKey := path.Join(key...)\n\tdb.cache[fullKey] = val\n\tdelete(db.deletes, fullKey)\n}\n\n// Clear removes all keys below and including `key`.\nfunc (db *DiskDatabase) Clear(key ...string) error {\n\tif debug {\n\t\tfmt.Println(\"CLEAR\", key)\n\t}\n\n\t// Cache the real modification for later:\n\tdb.ops = append(db.ops, func() error {\n\t\tfilePrefix := filepath.Join(db.basePath, fixDirectoryKeys(key))\n\t\twalker := func(path string, info os.FileInfo, err error) error {\n\t\t\tif os.IsNotExist(err) {\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tif !info.IsDir() {\n\t\t\t\treturn os.Remove(path)\n\t\t\t}\n\n\t\t\treturn nil\n\t\t}\n\t\treturn filepath.Walk(filePrefix, walker)\n\t})\n\n\t// Make sure we also modify the currently cached objects:\n\tprefix := path.Join(key...)\n\tfor key := range db.cache {\n\t\tif strings.HasPrefix(key, prefix) {\n\t\t\tdelete(db.cache, key)\n\t\t\tdb.deletes[key] = struct{}{}\n\t\t}\n\t}\n\n\t// Also check what keys we actually need to delete.\n\tfilePrefix := filepath.Join(db.basePath, fixDirectoryKeys(key))\n\twalker := func(filePath string, info os.FileInfo, err error) error {\n\t\tif os.IsNotExist(err) {\n\t\t\treturn nil\n\t\t}\n\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !info.IsDir() {\n\t\t\tkey := reverseDirectoryKeys(filePath[len(db.basePath):])\n\t\t\tdb.deletes[path.Join(key...)] = struct{}{}\n\t\t}\n\n\t\treturn nil\n\t}\n\n\treturn filepath.Walk(filePrefix, walker)\n}\n\n// Erase is the disk implementation of Database.Erase\nfunc (db *DiskDatabase) Erase(key ...string) {\n\tif debug {\n\t\tfmt.Println(\"ERASE\", key)\n\t}\n\n\tdb.ops = append(db.ops, func() error {\n\t\tfullPath := filepath.Join(db.basePath, fixDirectoryKeys(key))\n\t\terr := os.Remove(fullPath)\n\t\tif os.IsNotExist(err) {\n\t\t\treturn ErrNoSuchKey\n\t\t}\n\n\t\treturn err\n\t})\n\n\tfullKey := path.Join(key...)\n\tdb.deletes[fullKey] = struct{}{}\n\tdelete(db.cache, fullKey)\n}\n\n// HaveWrites is the disk implementation of Database.HaveWrites\nfunc (db *DiskDatabase) HaveWrites() bool {\n\treturn len(db.ops) > 0\n}\n\n// Keys is the disk implementation of Database.Keys\nfunc (db *DiskDatabase) Keys(prefix ...string) ([][]string, error) {\n\tfullPath := filepath.Join(db.basePath, fixDirectoryKeys(prefix))\n\tif _, err := os.Stat(fullPath); os.IsNotExist(err) {\n\t\treturn nil, nil\n\t}\n\n\tkeys := [][]string{}\n\treturn keys, filepath.Walk(fullPath, func(filePath string, info os.FileInfo, err error) error {\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !info.IsDir() {\n\t\t\tkey := reverseDirectoryKeys(filePath[len(db.basePath):])\n\t\t\tif _, ok := db.deletes[path.Join(key...)]; !ok {\n\t\t\t\tkeys = append(keys, key)\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t})\n}\n\n// Glob is the disk implementation of Database.Glob\nfunc (db *DiskDatabase) Glob(prefix []string) ([][]string, error) {\n\tfullPrefix := filepath.Join(db.basePath, filepath.Join(prefix...))\n\tmatches, err := filepath.Glob(fullPrefix + \"*\")\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tresults := [][]string{}\n\tfor _, match := range matches {\n\t\tinfo, err := os.Stat(match)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif !info.IsDir() {\n\t\t\tkey := match[len(db.basePath)+1:]\n\t\t\tif _, ok := db.deletes[key]; !ok {\n\t\t\t\tresults = append(results, strings.Split(key, string(filepath.Separator)))\n\t\t\t}\n\t\t}\n\t}\n\n\treturn results, nil\n}\n\n// Export writes all key/values into a gzipped .tar that is written to `w`.\nfunc (db *DiskDatabase) Export(w io.Writer) error {\n\tarchiveName := fmt.Sprintf(\"brigmeta-%s.gz\", time.Now().Format(time.RFC3339))\n\treturn util.Tar(db.basePath, archiveName, w)\n}\n\n// Import a gzipped tar from `r` into the current database.\nfunc (db *DiskDatabase) Import(r io.Reader) error {\n\treturn util.Untar(r, db.basePath)\n}\n\n// Close the database\nfunc (db *DiskDatabase) Close() error {\n\treturn nil\n}\n" }, { "path": "catfs/db/database_memory.go", "content": "package db\n\nimport (\n\t\"encoding/gob\"\n\t\"io\"\n\t\"path\"\n\t\"sort\"\n\t\"strings\"\n)\n\n// MemoryDatabase is a purely in memory database.\ntype MemoryDatabase struct {\n\tdata map[string][]byte\n\toldData map[string][]byte\n\thaveWrites bool\n\trefCount int\n}\n\n// a shallow copy is enough here.\nfunc shallowCopyMap(src map[string][]byte) map[string][]byte {\n\tdst := make(map[string][]byte)\n\tfor k, v := range src {\n\t\tcopyV := make([]byte, len(v))\n\t\tcopy(copyV, v)\n\t\tdst[k] = copyV\n\t}\n\n\treturn dst\n}\n\n// NewMemoryDatabase allocates a new empty MemoryDatabase\nfunc NewMemoryDatabase() *MemoryDatabase {\n\treturn &MemoryDatabase{\n\t\tdata: make(map[string][]byte),\n\t}\n}\n\n// Batch is a no-op for a memory database.\nfunc (mdb *MemoryDatabase) Batch() Batch {\n\tif mdb.refCount == 0 {\n\t\tmdb.oldData = shallowCopyMap(mdb.data)\n\t}\n\n\tmdb.refCount++\n\treturn mdb\n}\n\n// Flush is a no-op for a memory database.\nfunc (mdb *MemoryDatabase) Flush() error {\n\tmdb.refCount--\n\n\tif mdb.refCount == 0 {\n\t\tmdb.haveWrites = false\n\t}\n\treturn nil\n}\n\n// Rollback is a no-op for a memory database\nfunc (mdb *MemoryDatabase) Rollback() {\n\tif mdb.oldData != nil {\n\t\tmdb.data = shallowCopyMap(mdb.oldData)\n\t\tmdb.oldData = nil\n\t}\n\n\tmdb.refCount = 0\n}\n\n// Get returns `key` of `bucket`.\nfunc (mdb *MemoryDatabase) Get(key ...string) ([]byte, error) {\n\tdata, ok := mdb.data[path.Join(key...)]\n\tif !ok {\n\t\treturn nil, ErrNoSuchKey\n\t}\n\n\treturn data, nil\n}\n\n// Put sets `key` in `bucket` to `data`.\nfunc (mdb *MemoryDatabase) Put(data []byte, key ...string) {\n\tmdb.haveWrites = true\n\tmdb.data[path.Join(key...)] = data\n}\n\n// Clear removes all keys includin and below `key`.\nfunc (mdb *MemoryDatabase) Clear(key ...string) error {\n\tmdb.haveWrites = true\n\tjoinedKey := path.Join(key...)\n\tfor mapKey := range mdb.data {\n\t\tif strings.HasPrefix(mapKey, joinedKey) {\n\t\t\tdelete(mdb.data, mapKey)\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// Erase removes `key`\nfunc (mdb *MemoryDatabase) Erase(key ...string) {\n\tfullKey := path.Join(key...)\n\tmdb.haveWrites = true\n\tdelete(mdb.data, fullKey)\n}\n\n// Keys will return all keys currently stored in the memory map\nfunc (mdb *MemoryDatabase) Keys(prefix ...string) ([][]string, error) {\n\tkeys := [][]string{}\n\tfor key := range mdb.data {\n\t\tsplitKey := strings.Split(key, \"/\")\n\n\t\thasPrefix := len(prefix) <= len(splitKey)\n\t\tfor i := 0; hasPrefix && i < len(prefix) && i < len(splitKey); i++ {\n\t\t\tif prefix[i] != splitKey[i] {\n\t\t\t\thasPrefix = false\n\t\t\t}\n\t\t}\n\n\t\tif hasPrefix {\n\t\t\tkeys = append(keys, splitKey)\n\t\t}\n\n\t}\n\n\tsort.Slice(keys, func(i, j int) bool {\n\t\ta := strings.Join(keys[i], \".\")\n\t\tb := strings.Join(keys[j], \".\")\n\t\treturn a < b\n\t})\n\n\treturn keys, nil\n}\n\n// HaveWrites returns true if there are any open writes.\nfunc (mdb *MemoryDatabase) HaveWrites() bool {\n\treturn mdb.haveWrites\n}\n\n// Glob returns all keys starting with `prefix`.\nfunc (mdb *MemoryDatabase) Glob(prefix []string) ([][]string, error) {\n\tprefixKey := path.Join(prefix...)\n\n\tvar result [][]string\n\n\tkeys, err := mdb.Keys()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor _, key := range keys {\n\t\tfullKey := path.Join(key...)\n\t\tif strings.HasPrefix(fullKey, prefixKey) {\n\t\t\t// Filter \"directories\":\n\t\t\tsuffix := fullKey[len(prefixKey):]\n\t\t\tif !strings.Contains(suffix, \"/\") {\n\t\t\t\tresult = append(result, strings.Split(fullKey, \"/\"))\n\t\t\t}\n\t\t}\n\t}\n\n\tsort.Slice(result, func(i, j int) bool {\n\t\treturn path.Join(result[i]...) < path.Join(result[j]...)\n\t})\n\n\treturn result, nil\n}\n\n// Export encodes the internal memory map to a gob structure,\n// and writes it to `w`.\nfunc (mdb *MemoryDatabase) Export(w io.Writer) error {\n\treturn gob.NewEncoder(w).Encode(mdb.data)\n}\n\n// Import imports a previously exported dump and decodes the gob structure.\nfunc (mdb *MemoryDatabase) Import(r io.Reader) error {\n\treturn gob.NewDecoder(r).Decode(&mdb.data)\n}\n\n// Close the memory - a no op.\nfunc (mdb *MemoryDatabase) Close() error {\n\treturn nil\n}\n" }, { "path": "catfs/db/database_test.go", "content": "package db\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"sort\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/sahib/brig/util/testutil\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc withDiskDatabase(fn func(db *DiskDatabase)) error {\n\ttestDir, _ := ioutil.TempDir(\"\", \"brig-\")\n\tdefer os.RemoveAll(testDir)\n\n\tdb, err := NewDiskDatabase(testDir)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfn(db)\n\treturn db.Close()\n}\n\nfunc withBadgerDatabase(fn func(db *BadgerDatabase)) error {\n\ttestDir, _ := ioutil.TempDir(\"\", \"brig-\")\n\tdefer os.RemoveAll(testDir)\n\n\tdb, err := NewBadgerDatabase(testDir)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfn(db)\n\treturn db.Close()\n}\n\nfunc withMemDatabase(fn func(db *MemoryDatabase)) error {\n\tmdb := NewMemoryDatabase()\n\tfn(mdb)\n\treturn mdb.Close()\n}\n\nfunc withDbByName(name string, fn func(db Database)) error {\n\tswitch name {\n\tcase \"memory\":\n\t\treturn withMemDatabase(func(db *MemoryDatabase) {\n\t\t\tfn(db)\n\t\t})\n\tcase \"disk\":\n\t\treturn withDiskDatabase(func(db *DiskDatabase) {\n\t\t\tfn(db)\n\t\t})\n\tcase \"badger\":\n\t\treturn withBadgerDatabase(func(db *BadgerDatabase) {\n\t\t\tfn(db)\n\t\t})\n\tdefault:\n\t\tpanic(\"bad db name: \" + name)\n\t}\n}\n\nfunc withDbsByName(name string, fn func(db1, db2 Database)) error {\n\treturn withDbByName(name, func(db1 Database) {\n\t\twithDbByName(name, func(db2 Database) {\n\t\t\tfn(db1, db2)\n\t\t})\n\t})\n}\n\n//////////\n\nfunc TestDatabase(t *testing.T) {\n\tt.Run(\"memory\", func(t *testing.T) {\n\t\ttestDatabaseOneDb(t, \"memory\")\n\t\ttestDatabaseTwoDbs(t, \"memory\")\n\t})\n\tt.Run(\"disk\", func(t *testing.T) {\n\t\ttestDatabaseOneDb(t, \"disk\")\n\t\ttestDatabaseTwoDbs(t, \"disk\")\n\t})\n\tt.Run(\"badger\", func(t *testing.T) {\n\t\ttestDatabaseOneDb(t, \"badger\")\n\t\ttestDatabaseTwoDbs(t, \"badger\")\n\t})\n}\n\nfunc TestGCRace(t *testing.T) {\n\twithDbByName(\"badger\", func(db Database) {\n\t\tb := db.Batch()\n\t\tb.Put([]byte{0}, \"0\")\n\t\trequire.Nil(t, b.Flush())\n\n\t\tkeys, err := db.Keys()\n\t\trequire.Nil(t, err)\n\t\tfor _, key := range keys {\n\t\t\tb := db.Batch()\n\t\t\tb.Erase(key...)\n\t\t}\n\t})\n}\n\n//////////\n\nfunc testDatabaseTwoDbs(t *testing.T, name string) {\n\ttcs := []struct {\n\t\tname string\n\t\ttest func(t *testing.T, db1, db2 Database)\n\t}{\n\t\t{\n\t\t\tname: \"export-import\",\n\t\t\ttest: testExportImport,\n\t\t},\n\t}\n\n\tt.Run(\"double\", func(t *testing.T) {\n\t\tfor _, tc := range tcs {\n\t\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t\trequire.Nil(t, withDbsByName(name, func(db1, db2 Database) {\n\t\t\t\t\ttc.test(t, db1, db2)\n\t\t\t\t}))\n\t\t\t})\n\t\t}\n\t})\n}\n\nfunc testDatabaseOneDb(t *testing.T, name string) {\n\ttcs := []struct {\n\t\tname string\n\t\ttest func(t *testing.T, db Database)\n\t}{\n\t\t{\n\t\t\tname: \"put-and-get\",\n\t\t\ttest: testPutAndGet,\n\t\t}, {\n\t\t\tname: \"glob\",\n\t\t\ttest: testGlob,\n\t\t}, {\n\t\t\tname: \"clear\",\n\t\t\ttest: testClear,\n\t\t}, {\n\t\t\tname: \"clear-prefix\",\n\t\t\ttest: testClearPrefix,\n\t\t}, {\n\t\t\tname: \"invalid-access\",\n\t\t\ttest: testInvalidAccess,\n\t\t}, {\n\t\t\tname: \"recursive-batch\",\n\t\t\ttest: testRecursiveBatch,\n\t\t}, {\n\t\t\tname: \"rollback\",\n\t\t\ttest: testRollback,\n\t\t}, {\n\t\t\tname: \"erase\",\n\t\t\ttest: testErase,\n\t\t}, {\n\t\t\tname: \"keys\",\n\t\t\ttest: testKeys,\n\t\t},\n\t}\n\n\tt.Run(\"single\", func(t *testing.T) {\n\t\tfor _, tc := range tcs {\n\t\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t\trequire.Nil(t, withDbByName(name, func(db Database) {\n\t\t\t\t\ttc.test(t, db)\n\t\t\t\t}))\n\t\t\t})\n\t\t}\n\t})\n}\n\n///////////\n\nfunc testErase(t *testing.T, db Database) {\n\tbatch := db.Batch()\n\tbatch.Put([]byte{1}, \"existing_key\")\n\tbatch.Flush()\n\n\tbatch = db.Batch()\n\tbatch.Erase(\"existing_key\")\n\n\t_, err := db.Get(\"existing_key\")\n\trequire.Equal(t, ErrNoSuchKey, err)\n\n\tbatch.Flush()\n\n\t_, err = db.Get(\"existing_key\")\n\trequire.Equal(t, ErrNoSuchKey, err)\n}\n\nfunc testKeys(t *testing.T, db Database) {\n\tbatch := db.Batch()\n\texpect := [][]string{}\n\tfor i := 0; i < 15; i++ {\n\t\tkey := fmt.Sprintf(\"%d\", i)\n\t\tbatch.Put([]byte{byte(i)}, key)\n\t\texpect = append(expect, []string{key})\n\t}\n\tbatch.Flush()\n\n\tsort.Slice(expect, func(i, j int) bool {\n\t\ta := strings.Join(expect[i], \".\")\n\t\tb := strings.Join(expect[j], \".\")\n\t\treturn a < b\n\t})\n\n\textractKeys := func(prefixes []string) [][]string {\n\t\tkeys, err := db.Keys(prefixes...)\n\t\trequire.Nil(t, err)\n\t\treturn keys\n\t}\n\n\tkeys := extractKeys(nil)\n\trequire.Equal(t, expect, keys)\n\n\tkeys = extractKeys([]string{\"1\"})\n\trequire.Equal(t,\n\t\t[][]string{{\"1\"}},\n\t\tkeys,\n\t)\n}\n\nfunc testRollback(t *testing.T, db Database) {\n\tbatch := db.Batch()\n\tbatch.Put([]byte{1}, \"existing_key\")\n\tbatch.Flush()\n\n\tbatch = db.Batch()\n\tbatch.Put([]byte{2}, \"existing_key\")\n\tbatch.Put([]byte{2}, \"some_key\")\n\n\tdata, err := db.Get(\"some_key\")\n\trequire.Nil(t, err)\n\trequire.Equal(t, []byte{2}, data)\n\n\tbatch.Rollback()\n\n\tdata, err = db.Get(\"existing_key\")\n\trequire.Nil(t, err)\n\trequire.Equal(t, []byte{1}, data)\n\n\tdata, err = db.Get(\"some_key\")\n\trequire.Equal(t, ErrNoSuchKey, err)\n\trequire.Nil(t, data)\n}\n\nfunc testRecursiveBatch(t *testing.T, db Database) {\n\tbatch1 := db.Batch()\n\tbatch2 := db.Batch()\n\n\tbatch2.Put([]byte{1}, \"batch2_key\")\n\tval, err := db.Get(\"batch2_key\")\n\n\trequire.Nil(t, err)\n\trequire.Equal(t, []byte{1}, val)\n\n\trequire.True(t, batch1.HaveWrites())\n\trequire.True(t, batch2.HaveWrites())\n\trequire.Nil(t, batch2.Flush())\n\n\trequire.True(t, batch1.HaveWrites())\n\trequire.True(t, batch2.HaveWrites())\n\n\trequire.Nil(t, batch1.Flush())\n\trequire.False(t, batch1.HaveWrites())\n\trequire.False(t, batch2.HaveWrites())\n}\n\nfunc testPutAndGet(t *testing.T, db Database) {\n\ttestKeys := [][]string{\n\t\t{\"some\", \"stuff\", \"x\"},\n\t\t{\"some\", \"stuff\", \".\"},\n\t\t{\".\", \".\", \".\"},\n\t\t{\"some\", \"stuff\", \"__NO_DOT__\"},\n\t\t{\"some\", \"stuff\", \"DOT\"},\n\t}\n\n\tfor _, key := range testKeys {\n\t\tt.Run(strings.Join(key, \".\"), func(t *testing.T) {\n\t\t\tbatch := db.Batch()\n\t\t\tbatch.Put([]byte(\"hello\"), key...)\n\t\t\trequire.Nil(t, batch.Flush())\n\n\t\t\tdata, err := db.Get(key...)\n\t\t\trequire.Nil(t, err)\n\t\t\trequire.Equal(t, []byte(\"hello\"), data)\n\t\t})\n\t}\n}\n\nfunc testInvalidAccess(t *testing.T, db Database) {\n\tval, err := db.Get(\"hello\", \"world\")\n\trequire.Equal(t, ErrNoSuchKey, err)\n\trequire.Nil(t, val)\n}\n\nfunc testClear(t *testing.T, db Database) {\n\tbatch := db.Batch()\n\tfor i := 0; i < 100; i++ {\n\t\tbatch.Put([]byte{1}, \"a\", \"b\", \"c\", fmt.Sprintf(\"%d\", i))\n\t}\n\n\trequire.Nil(t, batch.Flush())\n\n\tbatch = db.Batch()\n\trequire.Nil(t, batch.Clear())\n\n\t// before flush:\n\tfor i := 0; i < 100; i++ {\n\t\t_, err := db.Get(\"a\", \"b\", \"c\", fmt.Sprintf(\"%d\", i))\n\t\trequire.Equal(t, ErrNoSuchKey, err)\n\t}\n\n\trequire.Nil(t, batch.Flush())\n\n\t// after flush:\n\tfor i := 0; i < 100; i++ {\n\t\t_, err := db.Get(\"a\", \"b\", \"c\", fmt.Sprintf(\"%d\", i))\n\t\trequire.Equal(t, ErrNoSuchKey, err)\n\t}\n}\n\nfunc testClearPrefix(t *testing.T, db Database) {\n\tbatch := db.Batch()\n\tfor i := 0; i < 10; i++ {\n\t\tbatch.Put([]byte{1}, \"a\", \"b\", \"c\", fmt.Sprintf(\"%d\", i))\n\t}\n\n\tfor i := 0; i < 10; i++ {\n\t\tbatch.Put([]byte{1}, \"x\", \"y\", \"z\", fmt.Sprintf(\"%d\", i))\n\t}\n\n\trequire.Nil(t, batch.Flush())\n\n\tbatch = db.Batch()\n\trequire.Nil(t, batch.Clear(\"a\"))\n\n\t// before flush:\n\tfor i := 0; i < 10; i++ {\n\t\t_, err := db.Get(\"a\", \"b\", \"c\", fmt.Sprintf(\"%d\", i))\n\t\trequire.Equal(t, ErrNoSuchKey, err)\n\t}\n\n\tfor i := 0; i < 10; i++ {\n\t\tdata, err := db.Get(\"x\", \"y\", \"z\", fmt.Sprintf(\"%d\", i))\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, []byte{1}, data)\n\t}\n\n\trequire.Nil(t, batch.Flush())\n\n\t// after flush:\n\tfor i := 0; i < 10; i++ {\n\t\t_, err := db.Get(\"a\", \"b\", \"c\", fmt.Sprintf(\"%d\", i))\n\t\trequire.Equal(t, ErrNoSuchKey, err)\n\t}\n\n\tfor i := 0; i < 10; i++ {\n\t\tdata, err := db.Get(\"x\", \"y\", \"z\", fmt.Sprintf(\"%d\", i))\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, []byte{1}, data)\n\t}\n}\n\nfunc testGlob(t *testing.T, db Database) {\n\tbatch := db.Batch()\n\tbatch.Put([]byte{1}, \"a\", \"b\", \"pref_1\")\n\tbatch.Put([]byte{2}, \"a\", \"b\", \"pref_2\")\n\tbatch.Put([]byte{3}, \"a\", \"b\", \"prev_3\")\n\tbatch.Put([]byte{4}, \"a\", \"b\", \"pref_dir\", \"x\")\n\n\terr := batch.Flush()\n\trequire.Nil(t, err)\n\n\tmatches, err := db.Glob([]string{\"a\", \"b\", \"pref_\"})\n\trequire.Nil(t, err)\n\n\trequire.Equal(t, [][]string{\n\t\t{\"a\", \"b\", \"pref_1\"},\n\t\t{\"a\", \"b\", \"pref_2\"},\n\t}, matches)\n}\n\nfunc testExportImport(t *testing.T, db1, db2 Database) {\n\ttestKeys := [][]string{\n\t\t{\"some\", \"stuff\", \"x\"},\n\t\t{\"some\", \"stuff\", \".\"},\n\t\t{\"some\", \"stuff\", \"__NO_DOT__\"},\n\t\t{\"some\", \"stuff\", \"DOT\"},\n\t}\n\n\tbatch := db1.Batch()\n\tfor _, key := range testKeys {\n\t\tbatch.Put([]byte{1, 2, 3}, key...)\n\t}\n\n\trequire.Nil(t, batch.Flush())\n\n\tfor _, key := range testKeys {\n\t\tdata, err := db1.Get(key...)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, []byte{1, 2, 3}, data)\n\t}\n\n\tbuf := &bytes.Buffer{}\n\trequire.Nil(t, db1.Export(buf))\n\trequire.Nil(t, db2.Import(buf))\n\n\tfor _, key := range testKeys {\n\t\tdata, err := db2.Get(key...)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, []byte{1, 2, 3}, data)\n\t}\n}\n\n// Regression bug fix: too many key/values in a transaction\n// will cause badger to return ErrTxnTooBig, which should\n// be handled as retry. This code triggers this.\nfunc TestLargeBatch(t *testing.T) {\n\tnKeys := 1000 * 10\n\n\tfor _, name := range []string{\"badger\", \"memory\", \"disk\"} {\n\t\tt.Run(name, func(t *testing.T) {\n\t\t\twithDbByName(name, func(db Database) {\n\t\t\t\tbatch := db.Batch()\n\t\t\t\tfor idx := 0; idx < nKeys; idx++ {\n\t\t\t\t\tval := testutil.CreateRandomDummyBuf(int64(1024), int64(idx))\n\t\t\t\t\tkey := fmt.Sprintf(\"idx-%d\", idx)\n\t\t\t\t\tbatch.Put(val, key)\n\t\t\t\t}\n\n\t\t\t\trequire.Nil(t, batch.Flush())\n\t\t\t\tfor idx := 0; idx < nKeys; idx++ {\n\t\t\t\t\texpect := testutil.CreateRandomDummyBuf(int64(1024), int64(idx))\n\t\t\t\t\tkey := fmt.Sprintf(\"idx-%d\", idx)\n\t\t\t\t\tgot, err := db.Get(key)\n\t\t\t\t\trequire.Nil(t, err)\n\t\t\t\t\trequire.Equal(t, expect, got)\n\t\t\t\t}\n\t\t\t})\n\t\t})\n\t}\n}\n\nfunc BenchmarkDatabase(b *testing.B) {\n\tbenchmarks := map[string]func(*testing.B, Database){\n\t\t\"put\": benchmarkDatabasePut,\n\t\t\"get\": benchmarkDatabaseGet,\n\t}\n\n\tfor benchmarkName, benchmark := range benchmarks {\n\t\tb.Run(benchmarkName, func(b *testing.B) {\n\t\t\tfor _, name := range []string{\"badger\", \"memory\", \"disk\"} {\n\t\t\t\tb.Run(name, func(b *testing.B) {\n\t\t\t\t\twithDbByName(name, func(db Database) {\n\t\t\t\t\t\tb.ResetTimer()\n\t\t\t\t\t\tbenchmark(b, db)\n\t\t\t\t\t\tb.StopTimer()\n\t\t\t\t\t})\n\t\t\t\t})\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc benchmarkDatabasePut(b *testing.B, db Database) {\n\tbatch := db.Batch()\n\tfor i := 0; i < b.N; i++ {\n\t\tkeyName := fmt.Sprintf(\"key_%d\", i%(1024*1024))\n\t\tbatch.Put(testutil.CreateDummyBuf(128), keyName)\n\t}\n\tbatch.Flush()\n}\n\nfunc benchmarkDatabaseGet(b *testing.B, db Database) {\n\tbatch := db.Batch()\n\tfor i := 0; i < b.N; i++ {\n\t\tkeyName := fmt.Sprintf(\"key_%d\", i%(1024*1024))\n\t\tbatch.Put(testutil.CreateDummyBuf(128), \"prefix\", keyName)\n\t}\n\tbatch.Flush()\n\n\tb.ResetTimer()\n\tfor i := 0; i < b.N; i++ {\n\t\tkeyName := fmt.Sprintf(\"key_%d\", i%(1024*1024))\n\t\tdb.Get(\"prefix\", keyName)\n\t}\n}\n" }, { "path": "catfs/errors/errors.go", "content": "package catfs\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n)\n\nvar (\n\t// ErrStageNotEmpty is returned by Reset() when it was called without force.\n\t// and there are still changes in the staging area.\n\tErrStageNotEmpty = errors.New(\"there are changes in the staging area; use the --force\")\n\n\t// ErrNoChange is returned when trying to commit, but there is no change.\n\tErrNoChange = errors.New(\"nothing changed between the given versions\")\n\n\t// ErrAmbigiousRev is returned when a ref string could mean several commits.\n\tErrAmbigiousRev = errors.New(\"there is more than one rev with this prefix\")\n\n\t// ErrExists is returned if a node already exists at a path, but should not.\n\tErrExists = errors.New(\"File exists\")\n\n\t// ErrBadNode is returned when a wrong node type was passed to a method.\n\tErrBadNode = errors.New(\"Cannot convert to concrete type. Broken input data?\")\n)\n\n//////////////\n\n// ErrNoSuchRef is returned when a bad ref was used\ntype ErrNoSuchRef string\n\nfunc (e ErrNoSuchRef) Error() string {\n\treturn fmt.Sprintf(\"No ref found named `%s`\", string(e))\n}\n\n// IsErrNoSuchRef checks if `err` is a no such ref error.\nfunc IsErrNoSuchRef(err error) bool {\n\t_, ok := err.(ErrNoSuchRef)\n\treturn ok\n}\n\n/////////////////\n\n// ErrNoSuchCommitIndex is returned when a bad commit was used\ntype ErrNoSuchCommitIndex struct {\n\tindex int64\n}\n\nfunc (e ErrNoSuchCommitIndex) Error() string {\n\treturn fmt.Sprintf(\"No commit with index `%d` found\", e.index)\n}\n\n// NoSuchCommitIndex returns an error for a missing index at `ind`\nfunc NoSuchCommitIndex(ind int64) error {\n\treturn &ErrNoSuchCommitIndex{ind}\n}\n\n// IsErrNoSuchCommitIndex checks if `err` is a no such ref error.\nfunc IsErrNoSuchCommitIndex(err error) bool {\n\t_, ok := err.(*ErrNoSuchCommitIndex)\n\treturn ok\n}\n\n/////////////////\n\ntype errNoSuchFile struct {\n\tpath string\n}\n\n// Error will return an error description detailin what path is missing.\nfunc (e *errNoSuchFile) Error() string {\n\treturn \"No such file or directory: \" + e.path\n}\n\n//////////////\n\n// NoSuchFile creates a new error that reports `path` as missing\nfunc NoSuchFile(path string) error {\n\treturn &errNoSuchFile{path}\n}\n\n// IsNoSuchFileError asserts that `err` means that the file could not be found\nfunc IsNoSuchFileError(err error) bool {\n\t_, ok := err.(*errNoSuchFile)\n\treturn ok\n}\n" }, { "path": "catfs/fs.go", "content": "package catfs\n\nimport (\n\t\"archive/tar\"\n\t\"bytes\"\n\t\"crypto/rand\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"path\"\n\t\"sort\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/sahib/config\"\n\tlog \"github.com/sirupsen/logrus\"\n\tcapnp \"zombiezen.com/go/capnproto2\"\n\n\te \"github.com/pkg/errors\"\n\tc \"github.com/sahib/brig/catfs/core\"\n\t\"github.com/sahib/brig/catfs/db\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\t\"github.com/sahib/brig/catfs/mio\"\n\t\"github.com/sahib/brig/catfs/mio/pagecache\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\t\"github.com/sahib/brig/catfs/vcs\"\n\t\"github.com/sahib/brig/repo/hints\"\n\t\"github.com/sahib/brig/util\"\n\th \"github.com/sahib/brig/util/hashlib\"\n)\n\nconst (\n\tabiVersion = 1\n\tdefaultEncryptionKeyLength = 32\n)\n\nfunc emptyFileEncryptionKey() []byte {\n\treturn make([]byte, defaultEncryptionKeyLength)\n}\n\n// HintManager is the API for looking up hints.\ntype HintManager interface {\n\t// Lookup should return stream hints for the path.\n\t// Hints are recursive, so we iterate until the root path\n\t// to find the correct hint.\n\tLookup(path string) hints.Hint\n\n\t// Set should remember `hint` for `path` and below.\n\tSet(path string, hint hints.Hint) error\n}\n\n// dummy hint manager that will always yield the default.\ntype defaultHintManager struct{}\n\nfunc (dhm defaultHintManager) Lookup(path string) hints.Hint {\n\treturn hints.Default()\n}\n\nfunc (dhm defaultHintManager) Set(path string, hint hints.Hint) error {\n\treturn fmt.Errorf(\"no hint manager, cannot remember hints\")\n}\n\n// FS (short for Filesystem) is the central API entry for everything related to\n// paths. It exposes a POSIX-like interface where path are mapped to the\n// actual underlying hashes and the associated metadata.\n//\n// Additionally it supports version control commands like MakeCommit(),\n// Checkout() etc. The API is file-centric, i.e. directories are created on\n// the fly for some operations like Stage(). Empty directories can be created\n// via Mkdir() though.\ntype FS struct {\n\tmu sync.Mutex\n\n\t// underlying key/value store\n\tkv db.Database\n\n\t// linker (holds all nodes together)\n\tlkr *c.Linker\n\n\t// garbage collector for dead metadata links\n\tgc *c.GarbageCollector\n\n\t// channel to schedule gc runs and quit the gc loop\n\tgcControl chan bool\n\n\t// channel to schedule auto commits and quit the loop\n\tautoCommitControl chan bool\n\n\t// channel to schedule repins and quit the loop\n\trepinControl chan string\n\n\t// Actual storage backend (e.g. ipfs or memory)\n\tbk FsBackend\n\n\t// internal config\n\tcfg *config.Config\n\n\t// cache for the isPinned operation\n\tpinner *Pinner\n\n\t// wether this fs is read only and cannot be changed.\n\t// It can be change by applying patches though.\n\treadOnly bool\n\n\t// interface to load stream hints\n\thintManager HintManager\n\n\t// cache for storing pages written to catfs.Handle\n\t// (may be nil if not used, e.g. for tests)\n\tpageCache pagecache.Cache\n}\n\n// ErrReadOnly is returned when a file system was created in read only mode\n// and a modifying operation was called on it.\nvar ErrReadOnly = errors.New(\"fs is read only\")\n\n// StatInfo describes the metadata of a single node.\n// The concept is comparable to the POSIX stat() call.\ntype StatInfo struct {\n\t// Path is the full path to the file\n\tPath string\n\n\t// TreeHash is the hash of the node in the DAG\n\tTreeHash h.Hash\n\t// ContentHash is the actual hash of the content\n\t// (used to test for content equality)\n\tContentHash h.Hash\n\t// BackendHash is the hash under which the file is reachable\n\t// in the backend.\n\tBackendHash h.Hash\n\n\t// User is the name of the user that modified this node last.\n\tUser string\n\t// Size in bytes\n\tSize uint64\n\t// Cached Size in bytes, i.e. size at backend\n\tCachedSize int64\n\t// Inode is a unique number specific to this node\n\tInode uint64\n\t// Depth is the hierarchy level inside of this node (root has 0)\n\tDepth int\n\t// ModTime is the last modification timestamp\n\tModTime time.Time\n\n\t// IsDir tells you if this node is a dir\n\tIsDir bool\n\t// IsPinned tells you if this node is pinned (either implicit or explicit)\n\tIsPinned bool\n\t// IsExplicit is true when the user pinned this node on purpose\n\tIsExplicit bool\n\n\t// IsRaw indicates if the stream associated with the file (if any)\n\t// was encoded by brig or can be consumed from ipfs directly.\n\tIsRaw bool\n\n\t// Key is the encryption key for the file.\n\tKey []byte\n}\n\n// DiffPair is a pair of nodes.\n// It is returned by MakeDiff(), where the source\n// is a node on the remote side and the dst node is\n// a node on our side.\ntype DiffPair struct {\n\tSrc StatInfo `json:\"src\"`\n\tDst StatInfo `json:\"dst\"`\n}\n\n// Diff is a list of things that changed between to commits\ntype Diff struct {\n\t// Added is a list of nodes that were added newly\n\tAdded []StatInfo `json:\"added\"`\n\n\t// Removed is a list of nodes that were removed\n\tRemoved []StatInfo `json:\"removed\"`\n\n\t// Ignored is a list of nodes that were not considered\n\tIgnored []StatInfo `json:\"ignored\"`\n\n\t// Missing is a list of nodes that the remoe side is missing\n\tMissing []StatInfo `json:\"missing\"`\n\n\t// Moved is a list of nodes that changed path\n\tMoved []DiffPair `json:\"moved\"`\n\n\t// Merged is a list of nodes that can be merged automatically\n\tMerged []DiffPair `json:\"merged\"`\n\n\t// Conflict is a list of nodes that cannot be merged automatically\n\tConflict []DiffPair `json:\"conflict\"`\n}\n\n// Commit gives information about a single commit.\ntype Commit struct {\n\t// Hash is the id of this commit\n\tHash h.Hash\n\t// Msg describes the committed contents\n\tMsg string\n\t// Tags is a user defined list of tags\n\t// (tags like HEAD, CURR and INIT are assigned dynamically as exception)\n\tTags []string\n\t// Date is the time when the commit was made\n\tDate time.Time\n\t// Index is the index of the commit:\n\tIndex int64\n}\n\n// Change describes a single change to a node between two versions\ntype Change struct {\n\t// Path is the node that was changed\n\tPath string\n\n\t// IsPinned tells you if the content is pinned at this stage\n\tIsPinned bool\n\n\t// IsExplicty tells you if the content is pinned explicitly.\n\tIsExplicit bool\n\n\t// Change describes what was changed\n\tChange string\n\n\t// MovedTo indicates that the node at this Path was moved to\n\t// another location and that there is no node at this location now.\n\tMovedTo string\n\n\t// WasPreviouslyAt is filled when the node was moved\n\t// and was previously at another location.\n\tWasPreviouslyAt string\n\n\t// Head is the commit after the change\n\tHead *Commit\n\n\t// Next is the commit before the change\n\tNext *Commit\n}\n\n// ExplicitPin is a pair of path and commit id.\ntype ExplicitPin struct {\n\tPath string\n\tCommit string\n}\n\n/////////////////////\n// UTILITY HELPERS //\n/////////////////////\n\nfunc (fs *FS) nodeToStat(nd n.Node) *StatInfo {\n\tisPinned, isExplicit, err := fs.pinner.IsNodePinned(nd)\n\tif err != nil {\n\t\tlog.Warningf(\"stat: failed to acquire pin state: %v\", err)\n\t}\n\n\tvar isDir bool\n\tvar isRaw bool\n\tvar key []byte\n\n\tswitch nd.Type() {\n\tcase n.NodeTypeFile:\n\t\tfile, ok := nd.(*n.File)\n\t\tif ok {\n\t\t\tkey = make([]byte, len(file.Key()))\n\t\t\tcopy(key, file.Key())\n\t\t}\n\n\t\tisRaw = file.IsRaw()\n\tcase n.NodeTypeDirectory:\n\t\tisDir = true\n\tcase n.NodeTypeGhost:\n\t\tghost, ok := nd.(*n.Ghost)\n\t\tif ok {\n\t\t\tisDir = (ghost.OldNode().Type() == n.NodeTypeDirectory)\n\t\t}\n\t}\n\n\treturn &StatInfo{\n\t\tPath: nd.Path(),\n\t\tUser: nd.User(),\n\t\tModTime: nd.ModTime(),\n\t\tIsDir: isDir,\n\t\tInode: nd.Inode(),\n\t\tSize: nd.Size(),\n\t\tCachedSize: nd.CachedSize(),\n\t\tDepth: n.Depth(nd),\n\t\tIsPinned: isPinned,\n\t\tIsExplicit: isExplicit,\n\t\tIsRaw: isRaw,\n\t\tContentHash: nd.ContentHash().Clone(),\n\t\tBackendHash: nd.BackendHash().Clone(),\n\t\tTreeHash: nd.TreeHash().Clone(),\n\t\tKey: key,\n\t}\n}\n\nfunc lookupFileOrDir(lkr *c.Linker, path string) (n.ModNode, error) {\n\tnd, err := lkr.LookupNode(path)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif nd == nil || nd.Type() == n.NodeTypeGhost {\n\t\treturn nil, ie.NoSuchFile(path)\n\t}\n\n\tmodNd, ok := nd.(n.ModNode)\n\tif !ok {\n\t\treturn nil, ie.ErrBadNode\n\t}\n\n\treturn modNd, nil\n}\n\nfunc (fs *FS) handleGcEvent(nd n.Node) bool {\n\tif nd.Type() != n.NodeTypeFile {\n\t\treturn true\n\t}\n\n\tfile, ok := nd.(*n.File)\n\tif !ok {\n\t\treturn true\n\t}\n\n\tcontent := file.BackendHash()\n\tlog.Infof(\"unpinning gc'd node %v\", content.B58String())\n\n\t// This node will not be reachable anymore by brig.\n\t// Make sure it is also unpinned to save space.\n\tif err := fs.pinner.Unpin(file.Inode(), file.BackendHash(), true); err != nil {\n\t\tlog.Warningf(\"unpinning attempt failed: %v\", err)\n\t}\n\n\t// Still return true, no need to stop the GC\n\treturn true\n}\n\n///////////////////////////////\n// ACTUAL API IMPLEMENTATION //\n///////////////////////////////\n\nfunc (fs *FS) doGcRun() {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\towner, err := fs.lkr.Owner()\n\tif err != nil {\n\t\tlog.Warningf(\"gc: failed to get owner: %v\", err)\n\t\treturn\n\t}\n\n\tlog.Debugf(\"filesystem GC (for %s): running\", owner)\n\tif err := fs.gc.Run(true); err != nil {\n\t\tlog.Warnf(\"failed to run GC: %v\", err)\n\t}\n}\n\n// NewFilesystem creates a new CATFS filesystem.\n// This filesystem stores all its data in a Merkle DAG and is fully versioned.\nfunc NewFilesystem(\n\tbackend FsBackend,\n\tdbPath string,\n\towner string,\n\treadOnly bool,\n\tfsCfg *config.Config,\n\thintManager HintManager,\n\tpageCache pagecache.Cache,\n) (*FS, error) {\n\tkv, err := db.NewBadgerDatabase(dbPath)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tlkr := c.NewLinker(kv)\n\tif err := lkr.SetOwner(owner); err != nil {\n\t\treturn nil, err\n\t}\n\n\t// NOTE: This is the place to start migrations in the future.\n\tif err := lkr.SetABIVersion(abiVersion); err != nil {\n\t\treturn nil, err\n\t}\n\n\tpinCache, err := NewPinner(lkr, backend)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif hintManager == nil {\n\t\thintManager = defaultHintManager{}\n\t}\n\n\t// NOTE: We do not need to validate fsCfg here.\n\t// This is already done on the side of our config module.\n\t// (we just need to convert a few keys to the vcs.SyncOptions enum later).\n\n\tfs := &FS{\n\t\tkv: kv,\n\t\tlkr: lkr,\n\t\tbk: backend,\n\t\tcfg: fsCfg,\n\t\treadOnly: readOnly,\n\t\tgcControl: make(chan bool, 1),\n\t\tautoCommitControl: make(chan bool, 1),\n\t\trepinControl: make(chan string, 1),\n\t\tpinner: pinCache,\n\t\thintManager: hintManager,\n\t\tpageCache: pageCache,\n\t}\n\n\t// Start the garbage collection background task.\n\t// It will run locked every few seconds and removes unreachable\n\t// objects from the staging area.\n\tfs.gc = c.NewGarbageCollector(lkr, kv, fs.handleGcEvent)\n\n\tgo fs.gcLoop()\n\tgo fs.autoCommitLoop()\n\tgo fs.repinLoop()\n\n\treturn fs, nil\n}\n\nfunc (fs *FS) gcLoop() {\n\tgcTicker := time.NewTicker(120 * time.Second)\n\tdefer gcTicker.Stop()\n\tfor {\n\t\tselect {\n\t\tcase state := <-fs.gcControl:\n\t\t\tif state {\n\t\t\t\tfs.doGcRun()\n\t\t\t} else {\n\t\t\t\t// Quit the gc loop:\n\t\t\t\tlog.Debugf(\"quitting the GC loop\")\n\t\t\t\treturn\n\t\t\t}\n\t\tcase <-gcTicker.C:\n\t\t\tfs.doGcRun()\n\t\t}\n\t}\n}\n\nfunc (fs *FS) autoCommitLoop() {\n\tlastCheck := time.Now()\n\tcheckTicker := time.NewTicker(1 * time.Second)\n\tdefer checkTicker.Stop()\n\n\tfor {\n\t\tselect {\n\t\tcase <-fs.autoCommitControl:\n\t\t\tlog.Debugf(\"quitting the auto commit loop\")\n\t\t\treturn\n\t\tcase <-checkTicker.C:\n\t\t\tisEnabled := fs.cfg.Bool(\"autocommit.enabled\")\n\t\t\tif !isEnabled {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tif time.Since(lastCheck) >= fs.cfg.Duration(\"autocommit.interval\") {\n\t\t\t\tlastCheck = time.Now()\n\t\t\t\tmsg := fmt.Sprintf(\"auto commit at »%s«\", time.Now().Format(time.RFC822))\n\t\t\t\tif err := fs.MakeCommit(msg); err != nil && err != ie.ErrNoChange {\n\t\t\t\t\tlog.Warningf(\"failed to create auto commit: %v\", err)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc (fs *FS) repinLoop() {\n\tif fs.readOnly {\n\t\treturn\n\t}\n\n\tlastCheck := time.Now()\n\tcheckTicker := time.NewTicker(1 * time.Second)\n\tdefer checkTicker.Stop()\n\n\tfor {\n\t\tselect {\n\t\tcase root := <-fs.repinControl:\n\t\t\tif root == \"\" {\n\t\t\t\tlog.Debugf(\"quitting the repin loop\")\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\t// Execute a repin immediately otherwise.\n\t\t\t// (and reset the timer, so we don't get it twice)\n\t\t\tif err := fs.repin(root); err != nil {\n\t\t\t\tlog.Warningf(\"repin failed: %v\", err)\n\t\t\t}\n\n\t\t\tlastCheck = time.Now()\n\t\tcase <-checkTicker.C:\n\t\t\tisEnabled := fs.cfg.Bool(\"repin.enabled\")\n\t\t\tif !isEnabled {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tif time.Since(lastCheck) >= fs.cfg.Duration(\"repin.interval\") {\n\t\t\t\tlastCheck = time.Now()\n\n\t\t\t\tif err := fs.repin(\"/\"); err != nil {\n\t\t\t\t\tlog.Warningf(\"repin failed: %v\", err)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n}\n\n// Close will clean up internal storage.\nfunc (fs *FS) Close() error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tgo func() { fs.gcControl <- false }()\n\tgo func() { fs.autoCommitControl <- false }()\n\tgo func() { fs.repinControl <- \"\" }()\n\n\tif err := fs.pinner.Close(); err != nil {\n\t\tlog.Warnf(\"Failed to close pin cache: %v\", err)\n\t}\n\n\treturn fs.kv.Close()\n}\n\n// Export will export a serialized version of the filesystem to `w`.\nfunc (fs *FS) Export(w io.Writer) error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\treturn fs.kv.Export(w)\n}\n\n// Import will read a previously FS dump from `r`.\nfunc (fs *FS) Import(r io.Reader) error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tif err := fs.kv.Import(r); err != nil {\n\t\treturn err\n\t}\n\n\t// disk (probably) changed, delete memcache:\n\tfs.lkr.MemIndexClear()\n\treturn nil\n}\n\n/////////////////////\n// CORE OPERATIONS //\n/////////////////////\n\n// Move will move the file or directory at `src` to `dst`.\n// If it does not exist, an error will be returned.\nfunc (fs *FS) Move(src, dst string) error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tif fs.readOnly {\n\t\treturn ErrReadOnly\n\t}\n\n\tsrcNd, err := lookupFileOrDir(fs.lkr, src)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn c.Move(fs.lkr, srcNd, dst)\n}\n\n// Copy will copy the file or directory at `src` to `dst`.\n// If it does not exist, an error will be returned.\nfunc (fs *FS) Copy(src, dst string) error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tif fs.readOnly {\n\t\treturn ErrReadOnly\n\t}\n\n\tsrcNd, err := lookupFileOrDir(fs.lkr, src)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t_, err = c.Copy(fs.lkr, srcNd, dst)\n\treturn err\n}\n\n// Mkdir creates a new empty directory at `dir`, possibly creating\n// all intermediate parents if `createParents` is set.\nfunc (fs *FS) Mkdir(dir string, createParents bool) error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tif fs.readOnly {\n\t\treturn ErrReadOnly\n\t}\n\n\t// \"brig mkdir .\" somehow is able to overwrite everything:\n\tdir = strings.TrimLeft(path.Clean(dir), \".\")\n\t_, err := c.Mkdir(fs.lkr, dir, createParents)\n\treturn err\n}\n\n// Remove removes the file or directory at `path`.\nfunc (fs *FS) Remove(path string) error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tif fs.readOnly {\n\t\treturn ErrReadOnly\n\t}\n\n\tnd, err := lookupFileOrDir(fs.lkr, path)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// TODO: What should remove do with the pin state?\n\t_, _, err = c.Remove(fs.lkr, nd, true, true)\n\treturn err\n}\n\n// Stat delivers detailed information about the node at `path`.\nfunc (fs *FS) Stat(path string) (*StatInfo, error) {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tnd, err := fs.lkr.LookupNode(path)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif nd.Type() == n.NodeTypeGhost {\n\t\treturn nil, ie.NoSuchFile(path)\n\t}\n\n\treturn fs.nodeToStat(nd), nil\n}\n\n// Filter implements a quick and easy way to search over all files\n// by using a query that checks if it is part of the path.\nfunc (fs *FS) Filter(root, query string) ([]*StatInfo, error) {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\trootNd, err := fs.lkr.LookupNode(root)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif rootNd.Type() == n.NodeTypeGhost {\n\t\treturn nil, ie.NoSuchFile(root)\n\t}\n\n\tquery = strings.ToLower(query)\n\tresult := []*StatInfo{}\n\terr = n.Walk(fs.lkr, rootNd, false, func(child n.Node) error {\n\t\t// Ghost nodes should not be visible to the outside.\n\t\tif child.Type() == n.NodeTypeGhost {\n\t\t\treturn nil\n\t\t}\n\n\t\t// Special case: Forget about the root node.\n\t\t// It should not be part of the results.\n\t\tchildPath := child.Path()\n\t\tif childPath == root {\n\t\t\treturn nil\n\t\t}\n\n\t\tchildPath = strings.ToLower(childPath[len(root):])\n\t\tif !strings.Contains(childPath, query) {\n\t\t\treturn nil\n\t\t}\n\n\t\tresult = append(result, fs.nodeToStat(child))\n\t\treturn n.ErrSkipChild\n\t})\n\n\tsort.Slice(result, func(i, j int) bool {\n\t\tiDepth := result[i].Depth\n\t\tjDepth := result[j].Depth\n\n\t\tif iDepth == jDepth {\n\t\t\treturn result[i].Path < result[j].Path\n\t\t}\n\n\t\treturn iDepth < jDepth\n\t})\n\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn result, nil\n}\n\n// List returns stat info for each node below (and including) root.\n// Nodes deeper than maxDepth will not be shown. If maxDepth is a\n// negative number, all nodes will be shown.\nfunc (fs *FS) List(root string, maxDepth int) ([]*StatInfo, error) {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\t// NOTE: This method is highly inefficient:\n\t// - iterates over all nodes even if maxDepth is >= 0\n\t//\n\t// Fix whenever it proves to be a problem.\n\t// I don't want to engineer something now until I know what's needed.\n\trootNd, err := fs.lkr.LookupNode(root)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif rootNd.Type() == n.NodeTypeGhost {\n\t\treturn nil, ie.NoSuchFile(root)\n\t}\n\n\t// Start counting max depth relative to the root:\n\tif maxDepth >= 0 {\n\t\tmaxDepth += n.Depth(rootNd)\n\t}\n\n\tresult := []*StatInfo{}\n\tif rootNd.Type() == n.NodeTypeFile {\n\t\t// There is no point to Walk through file, it has no children\n\t\t// but we need to report on itself\n\t\tresult = append(result, fs.nodeToStat(rootNd))\n\t\treturn result, nil\n\t}\n\terr = n.Walk(fs.lkr, rootNd, false, func(child n.Node) error {\n\t\tif maxDepth < 0 || n.Depth(child) <= maxDepth {\n\t\t\tif maxDepth >= 0 && child.Path() == root {\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\t// Ghost nodes should not be visible to the outside.\n\t\t\tif child.Type() == n.NodeTypeGhost {\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\tresult = append(result, fs.nodeToStat(child))\n\t\t}\n\n\t\treturn nil\n\t})\n\n\tsort.Slice(result, func(i, j int) bool {\n\t\tiDepth := result[i].Depth\n\t\tjDepth := result[j].Depth\n\n\t\tif iDepth == jDepth {\n\t\t\treturn result[i].Path < result[j].Path\n\t\t}\n\n\t\treturn iDepth < jDepth\n\t})\n\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn result, nil\n}\n\n////////////////////////\n// PINNING OPERATIONS //\n////////////////////////\n\n// preCache makes the backend fetch the data already from the network,\n// even though it might not be needed yet.\nfunc (fs *FS) preCache(hash h.Hash) error {\n\tstream, err := fs.bk.Cat(hash)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t_, err = io.Copy(ioutil.Discard, stream)\n\treturn err\n}\n\nfunc (fs *FS) preCacheInBackground(hash h.Hash) {\n\tif !fs.cfg.Bool(\"pre_cache.enabled\") {\n\t\treturn\n\t}\n\n\tgo func() {\n\t\tif err := fs.preCache(hash); err != nil {\n\t\t\tlog.Debugf(\"failed to pre-cache `%s`: %v\", hash, err)\n\t\t}\n\t}()\n}\n\n// Pin will pin the file or directory at `path` explicitly.\nfunc (fs *FS) Pin(path, rev string, explicit bool) error {\n\treturn fs.doPin(path, rev, fs.pinner.PinNode, explicit)\n}\n\n// Unpin will unpin the file or directory at `path` explicitly.\nfunc (fs *FS) Unpin(path, rev string, explicit bool) error {\n\treturn fs.doPin(path, rev, fs.pinner.UnpinNode, explicit)\n}\n\nfunc (fs *FS) doPin(path, rev string, op func(nd n.Node, explicit bool) error, explicit bool) error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tcmt, err := parseRev(fs.lkr, rev)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\troot, err := fs.lkr.DirectoryByHash(cmt.Root())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tnd, err := root.Lookup(fs.lkr, path)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif nd == nil || nd.Type() == n.NodeTypeGhost {\n\t\treturn ie.NoSuchFile(path)\n\t}\n\n\tif err := op(nd, explicit); err != nil {\n\t\treturn err\n\t}\n\n\t// Make sure the data is available (if requested):\n\tif nd.Type() == n.NodeTypeFile {\n\t\tfs.preCacheInBackground(nd.BackendHash())\n\t}\n\n\treturn nil\n}\n\n// IsPinned returns true for files and directories that are pinned.\n// A directory only counts as pinned if all files and directories\n// in it are also pinned.\nfunc (fs *FS) IsPinned(path string) (bool, bool, error) {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tnd, err := lookupFileOrDir(fs.lkr, path)\n\tif err != nil {\n\t\treturn false, false, err\n\t}\n\n\treturn fs.pinner.IsNodePinned(nd)\n}\n\n////////////////////////\n// STAGING OPERATIONS //\n////////////////////////\n\nfunc prefixSlash(s string) string {\n\tif !strings.HasPrefix(s, \"/\") {\n\t\treturn \"/\" + s\n\t}\n\n\treturn s\n}\n\n// Touch creates an empty file at `path` if it does not exist yet.\n// If it exists, it's mod time is being updated to the current time.\nfunc (fs *FS) Touch(path string) error {\n\tfs.mu.Lock()\n\n\tif fs.readOnly {\n\t\tfs.mu.Unlock()\n\t\treturn ErrReadOnly\n\t}\n\n\tnd, err := fs.lkr.LookupNode(path)\n\tif err != nil && !ie.IsNoSuchFileError(err) {\n\t\tfs.mu.Unlock()\n\t\treturn err\n\t}\n\n\tif nd != nil {\n\t\tmodNd, ok := nd.(n.ModNode)\n\t\tif !ok {\n\t\t\t// What could it be if lookup returns not a node?\n\t\t\tfs.mu.Unlock()\n\t\t\treturn nil\n\t\t}\n\t\tif modNd.Type() != n.NodeTypeGhost {\n\t\t\tmodNd.SetModTime(time.Now())\n\t\t\tfs.mu.Unlock()\n\t\t\treturn nil\n\t\t}\n\t}\n\n\t// We may not call Stage() with a lock.\n\tfs.mu.Unlock()\n\n\t// Nothing or a ghost there, stage an empty file\n\t// 0 sized (newly touched) files should have the same key\n\t// to point to the same backend file\n\tkey := emptyFileEncryptionKey()\n\treturn fs.stageWithKey(prefixSlash(path), bytes.NewReader([]byte{}), key)\n}\n\n// Truncate cuts of the output of the file at `path` to `size`.\n// `size` should be between 0 and the size of the file,\n// all other values will be ignored.\n//\n// Note that this is not implemented as an actual IO operation.\n// It is possible to go back to a bigger size until the actual\n// content was changed via Stage().\nfunc (fs *FS) Truncate(path string, size uint64) error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tif fs.readOnly {\n\t\treturn ErrReadOnly\n\t}\n\n\tnd, err := fs.lkr.LookupModNode(path)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif nd.Type() != n.NodeTypeFile {\n\t\treturn fmt.Errorf(\"`%s` is not a file\", path)\n\t}\n\n\tnd.SetSize(size)\n\treturn fs.lkr.StageNode(nd)\n}\n\nfunc (fs *FS) renewPins(oldFile, newFile *n.File) error {\n\tpinExplicit := false\n\n\tif oldFile != nil {\n\t\toldBackendHash := oldFile.BackendHash()\n\t\tif oldBackendHash.Equal(newFile.BackendHash()) {\n\t\t\t// Nothing changed, nothing to do...\n\t\t\treturn nil\n\t\t}\n\n\t\t_, isExplicit, err := fs.pinner.IsNodePinned(oldFile)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// If the old file was pinned explicitly, we should also pin\n\t\t// the new file explicitly to carry over that info.\n\t\tpinExplicit = isExplicit\n\n\t\tif !isExplicit {\n\t\t\tif err := fs.pinner.UnpinNode(oldFile, pinExplicit); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t}\n\n\treturn fs.pinner.PinNode(newFile, pinExplicit)\n}\n\nfunc (fs *FS) preStageKeyGen(path string) ([]byte, error) {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\t// See if we already have such a file.\n\t// If not we gonna need to generate new key for it\n\t// based on the content hash.\n\toldNode, err := fs.lkr.LookupNode(path)\n\n\t// Check that we're handling the right kind of node.\n\t// We should be able to add on-top of ghosts, but directories\n\t// are pointless as input.\n\tvar oldFile *n.File\n\tif err == nil {\n\t\tswitch oldNode.Type() {\n\t\tcase n.NodeTypeDirectory:\n\t\t\treturn nil, fmt.Errorf(\"Cannot stage over directory: %v\", path)\n\t\tcase n.NodeTypeGhost:\n\t\t\t// Act like there was no such node:\n\t\t\terr = ie.NoSuchFile(path)\n\t\tcase n.NodeTypeFile:\n\t\t\tvar ok bool\n\t\t\toldFile, ok = oldNode.(*n.File)\n\t\t\tif !ok {\n\t\t\t\treturn nil, ie.ErrBadNode\n\t\t\t}\n\t\t}\n\t}\n\n\tif err != nil && !ie.IsNoSuchFileError(err) {\n\t\treturn nil, err\n\t}\n\n\tif oldFile != nil && oldFile.Size() != 0 {\n\t\treturn oldFile.Key(), nil\n\t}\n\n\t// only create a new key for new files.\n\t// The key depends on the content hash and the size.\n\tkey := make([]byte, defaultEncryptionKeyLength)\n\tif _, err := rand.Read(key); err != nil {\n\t\treturn nil, e.Wrapf(err, \"failed to generate random key\")\n\t}\n\n\treturn key, nil\n}\n\n// Stage reads all data from `r` and stores as content of the node at `path`.\n// If `path` already exists, it will be updated.\nfunc (fs *FS) Stage(path string, r io.Reader) error {\n\tif fs.readOnly {\n\t\treturn ErrReadOnly\n\t}\n\tpath = prefixSlash(path)\n\tkey, err := fs.preStageKeyGen(path)\n\tif err != nil {\n\t\treturn err\n\t}\n\treturn fs.stageWithKey(path, r, key)\n}\n\n// stageWithKey reads all data from `r` and stores as content of the node at `path`.\n// It uses provided encryption key\n// If `path` already exists, it will be updated.\nfunc (fs *FS) stageWithKey(path string, r io.Reader, key []byte) error {\n\tif fs.readOnly {\n\t\treturn ErrReadOnly\n\t}\n\tpath = prefixSlash(path)\n\n\t// NOTE: fs.mu is not locked here since I/O can be done in parallel.\n\t// If you need locking, you can do it at the bottom of this method.\n\n\t// Branch off a part of the stream and pipe it through\n\t// a hash writer to compute the hash while reading the stream:\n\thashWriter := h.NewHashWriter()\n\thashReader := io.TeeReader(r, hashWriter)\n\n\t// Do the same with the size.\n\t// This actually measures the size of the stream and is\n\t// therefore guaranteed to find out the actual stream size.\n\tsizeAcc := &util.SizeAccumulator{}\n\tsizeReader := io.TeeReader(hashReader, sizeAcc)\n\n\thint := fs.hintManager.Lookup(path)\n\tstream, isRaw, err := mio.NewInStream(sizeReader, path, key, hint)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tbackendHash, err := fs.bk.Add(stream)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// The stream was consumed, we now know those attrs:\n\tsize := sizeAcc.Size()\n\tcontentHash := hashWriter.Finalize()\n\n\t// Lock it again for the metadata staging:\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tcachedSize, err := fs.bk.CachedSize(backendHash)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Remember the metadata:\n\tnewFile, err := c.Stage(\n\t\tfs.lkr,\n\t\tpath,\n\t\tcontentHash,\n\t\tbackendHash,\n\t\tsize,\n\t\tcachedSize,\n\t\tkey,\n\t\ttime.Now(),\n\t\tisRaw,\n\t)\n\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn fs.pinner.PinNode(newFile, false)\n}\n\n////////////////////\n// I/O OPERATIONS //\n////////////////////\n\ntype tarEntry struct {\n\tpath string\n\tsize int64\n\tstream mio.Stream\n}\n\nfunc (fs *FS) getTarableEntries(root string, filter func(node *StatInfo) bool) ([]tarEntry, string, error) {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\trootNd, err := fs.lkr.LookupNode(root)\n\tif err != nil {\n\t\treturn nil, \"\", err\n\t}\n\n\tentries := []tarEntry{}\n\terr = n.Walk(fs.lkr, rootNd, false, func(child n.Node) error {\n\t\tif filter != nil && rootNd.Path() != child.Path() {\n\t\t\t// Ask the API user if he wants this node in his archive:\n\t\t\tif !filter(fs.nodeToStat(child)) {\n\t\t\t\treturn n.ErrSkipChild\n\t\t\t}\n\t\t}\n\n\t\tif child.Type() != n.NodeTypeFile {\n\t\t\treturn nil\n\t\t}\n\n\t\tfile, ok := child.(*n.File)\n\t\tif !ok {\n\t\t\treturn ie.ErrBadNode\n\t\t}\n\n\t\tstream, err := fs.catHash(\n\t\t\tfile.BackendHash(),\n\t\t\tfile.Key(),\n\t\t\tfile.Size(),\n\t\t\tfile.IsRaw(),\n\t\t)\n\t\tif err != nil {\n\t\t\treturn e.Wrapf(err, \"failed to open stream for %s\", file.Path())\n\t\t}\n\n\t\tentries = append(entries, tarEntry{\n\t\t\tpath: child.Path(),\n\t\t\tsize: int64(child.Size()),\n\t\t\tstream: stream,\n\t\t})\n\t\treturn nil\n\t})\n\n\t// Make sure that the entries are served in lexicographical order.\n\tsort.Slice(entries, func(i, j int) bool {\n\t\treturn entries[i].path < entries[j].path\n\t})\n\n\tprefixPath := root\n\tif rootNd.Type() != n.NodeTypeDirectory {\n\t\tprefixPath = path.Dir(root)\n\t}\n\n\treturn entries, prefixPath, err\n}\n\n// Tar produces a tar archive from the file or directory at `root` and writes\n// the output to `w`. If you want compression, supply a gzip writer.\nfunc (fs *FS) Tar(root string, w io.Writer, filter func(node *StatInfo) bool) error {\n\t// getTarableEntries is locking fs.mu while it is running.\n\t// the rest of the code in this method should NOT use any nodes\n\t// or anything that is open to race conditions!\n\tentries, prefixPath, err := fs.getTarableEntries(root, filter)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\ttw := tar.NewWriter(w)\n\n\t// Make sure to close all remaining streams when any error happens.\n\t// Also clean up the tar writer. This might flush some data still.\n\t// The user of this API should not use `w` if an error happens.\n\tcleanup := func(idx int) {\n\t\tfor ; idx < len(entries); idx++ {\n\t\t\tentry := entries[idx]\n\t\t\tif err := entry.stream.Close(); err != nil {\n\t\t\t\tlog.Debugf(\"could not close stream: %v (file descriptor leak?)\", entry.path)\n\t\t\t}\n\t\t}\n\n\t\ttw.Close()\n\t}\n\n\tfor idx, entry := range entries {\n\t\thdr := &tar.Header{\n\t\t\tName: entry.path[len(prefixPath):],\n\t\t\tMode: 0600,\n\t\t\tSize: entry.size,\n\t\t}\n\n\t\tif err := tw.WriteHeader(hdr); err != nil {\n\t\t\tcleanup(idx)\n\t\t\treturn err\n\t\t}\n\n\t\tif _, err := io.Copy(tw, entry.stream); err != nil {\n\t\t\tcleanup(idx)\n\t\t\treturn err\n\t\t}\n\n\t\tif err := entry.stream.Close(); err != nil {\n\t\t\tcleanup(idx + 1)\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn tw.Close()\n}\n\n// Cat will open a file read-only and expose it's underlying data as stream.\n// If no such path is known or it was deleted, nil is returned as stream.\nfunc (fs *FS) Cat(path string) (mio.Stream, error) {\n\tfs.mu.Lock()\n\n\tfile, err := fs.lkr.LookupFile(path)\n\tif err == ie.ErrBadNode {\n\t\tfs.mu.Unlock()\n\t\treturn nil, ie.NoSuchFile(path)\n\t}\n\n\tif err != nil {\n\t\tfs.mu.Unlock()\n\t\treturn nil, err\n\t}\n\n\t// Copy all attributes, since accessing them beyond the lock might be racy.\n\tsize := file.Size()\n\tbackendHash := file.BackendHash().Clone()\n\tkey := make([]byte, len(file.Key()))\n\tisRaw := file.IsRaw()\n\tcopy(key, file.Key())\n\n\tfs.mu.Unlock()\n\n\treturn fs.catHash(backendHash, key, size, isRaw)\n}\n\n// NOTE: This method can be called without locking fs.mu!\nfunc (fs *FS) catHash(backendHash h.Hash, key []byte, size uint64, isRaw bool) (mio.Stream, error) {\n\trawStream, err := fs.bk.Cat(backendHash)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tstream, err := mio.NewOutStream(rawStream, isRaw, key)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Truncate stream to file size. Data stream might be bigger\n\t// for example when fuse decided to truncate the file, but\n\t// did not flush it already.\n\treturn mio.LimitStream(stream, size), nil\n}\n\n// Open returns a file like object that can be used for modifying a file in memory.\n// If you want to have seekable read-only stream, use Cat(), it has less overhead.\nfunc (fs *FS) Open(path string) (*Handle, error) {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tnd, err := fs.lkr.LookupNode(path)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfile, ok := nd.(*n.File)\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"Can only open files: %v\", path)\n\t}\n\n\treturn newHandle(fs, file, fs.readOnly), nil\n}\n\n////////////////////\n// VCS OPERATIONS //\n////////////////////\n\n// MakeCommit bundles all staged changes into one commit described by `msg`.\n// If no changes were made since the last call to MakeCommit() ErrNoConflict\n// is returned.\nfunc (fs *FS) MakeCommit(msg string) error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\towner, err := fs.lkr.Owner()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn fs.lkr.MakeCommit(owner, msg)\n}\n\nfunc (fs *FS) isMove(nd n.ModNode) (bool, error) {\n\tcmt, err := fs.lkr.Status()\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\twalker := vcs.NewHistoryWalker(fs.lkr, cmt, nd)\n\tfor walker.Next() {\n\t\tstate := walker.State()\n\t\tif state.Mask == vcs.ChangeTypeNone {\n\t\t\tcontinue\n\t\t}\n\n\t\tif state.Mask&vcs.ChangeTypeMove != 0 {\n\t\t\treturn true, nil\n\t\t}\n\n\t\treturn false, nil\n\t}\n\n\treturn false, nil\n}\n\n// DeletedNodes returns all nodes under `root` that were deleted.\n// This does not include files that were moved. Note that you\n// cannot pass the paths of those files to methods like Cat(),\n// since they will refuse to work on deleted files.\nfunc (fs *FS) DeletedNodes(root string) ([]*StatInfo, error) {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\trootNd, err := fs.lkr.LookupNode(root)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tnodes := []*StatInfo{}\n\terr = n.Walk(fs.lkr, rootNd, false, func(child n.Node) error {\n\t\tif child.Type() != n.NodeTypeGhost {\n\t\t\treturn nil\n\t\t}\n\n\t\tmodNd, ok := child.(n.ModNode)\n\t\tif !ok {\n\t\t\treturn ie.ErrBadNode\n\t\t}\n\n\t\tisMove, err := fs.isMove(modNd)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !isMove {\n\t\t\tnodes = append(nodes, fs.nodeToStat(modNd))\n\t\t}\n\n\t\treturn nil\n\t})\n\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn nodes, nil\n}\n\n// Undelete tries to recover a file or directory that was previously deleted.\n// This will fail when being called on a regular file or directory.\n// You can obtain deleted paths by using DeletedNodes()\nfunc (fs *FS) Undelete(root string) error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tif fs.readOnly {\n\t\treturn ErrReadOnly\n\t}\n\n\tif err := vcs.Undelete(fs.lkr, root); err != nil {\n\t\treturn err\n\t}\n\n\tnd, err := fs.lkr.LookupModNode(root)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn fs.pinner.PinNode(nd, false)\n}\n\n// Head translates the \"head\" symbol to a ref.\nfunc (fs *FS) Head() (string, error) {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\thead, err := fs.lkr.Head()\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\treturn head.TreeHash().B58String(), nil\n}\n\n// Curr translates the \"curr\" symbol to a ref.\nfunc (fs *FS) Curr() (string, error) {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tstatus, err := fs.lkr.Status()\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\treturn status.TreeHash().B58String(), nil\n}\n\nfunc commitToExternal(cmt *n.Commit, hashToRef map[string][]string) *Commit {\n\ttags := []string{}\n\tif hashToRef != nil {\n\t\ttags = hashToRef[cmt.TreeHash().B58String()]\n\t}\n\n\treturn &Commit{\n\t\tHash: cmt.TreeHash().Clone(),\n\t\tMsg: cmt.Message(),\n\t\tTags: tags,\n\t\tDate: cmt.ModTime(),\n\t\tIndex: cmt.Index(),\n\t}\n}\n\n// History returns all modifications of a node with one entry per commit.\nfunc (fs *FS) History(path string) ([]Change, error) {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tnd, err := fs.lkr.LookupModNode(path)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tstatus, err := fs.lkr.Status()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\thist, err := vcs.History(fs.lkr, nd, status, nil)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\thashToRef, err := fs.buildCommitHashToRefTable()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tentries := []Change{}\n\tfor _, change := range hist {\n\t\thead := commitToExternal(change.Head, hashToRef)\n\n\t\tvar next *Commit\n\t\tif change.Next != nil {\n\t\t\tnext = commitToExternal(change.Next, hashToRef)\n\t\t}\n\n\t\tisPinned, isExplicit, err := fs.pinner.IsNodePinned(change.Curr)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tentries = append(entries, Change{\n\t\t\tPath: change.Curr.Path(),\n\t\t\tChange: change.Mask.String(),\n\t\t\tIsPinned: isPinned,\n\t\t\tIsExplicit: isExplicit,\n\t\t\tHead: head,\n\t\t\tNext: next,\n\t\t\tMovedTo: change.MovedTo,\n\t\t\tWasPreviouslyAt: change.WasPreviouslyAt,\n\t\t})\n\t}\n\n\treturn entries, nil\n}\n\nfunc (fs *FS) buildSyncCfg() (*vcs.SyncOptions, error) {\n\t// Helper method to easily pin depending on a condition variable\n\tdoPinOrUnpin := func(doPin, explicit bool, nd n.ModNode) {\n\t\tfile, ok := nd.(*n.File)\n\t\tif !ok {\n\t\t\t// Non-files are simply ignored.\n\t\t\treturn\n\t\t}\n\n\t\top := fs.pinner.UnpinNode\n\t\topName := \"unpin\"\n\t\tif doPin {\n\t\t\top = fs.pinner.PinNode\n\t\t\topName = \"pin\"\n\t\t}\n\n\t\tif err := op(file, explicit); err != nil {\n\t\t\tlog.Warningf(\"Failed to %s (hash: %v)\", opName, file.BackendHash())\n\t\t}\n\t}\n\n\tconflictStrategy := vcs.ConflictStrategyFromString(\n\t\tfs.cfg.String(\"sync.conflict_strategy\"),\n\t)\n\n\tif conflictStrategy == vcs.ConflictStragetyUnknown {\n\t\treturn nil, fmt.Errorf(\"unknown conflict strategy: %v\", conflictStrategy)\n\t}\n\n\treturn &vcs.SyncOptions{\n\t\tConflictStrategy: conflictStrategy,\n\t\tIgnoreDeletes: fs.cfg.Bool(\"sync.ignore_removed\"),\n\t\tIgnoreMoves: fs.cfg.Bool(\"sync.ignore_moved\"),\n\t\tOnAdd: func(newNd n.ModNode) bool {\n\t\t\tif fs.cfg.Bool(\"sync.pin_added\") {\n\t\t\t\t// do pinning and more importantly caching\n\t\t\t\tdoPinOrUnpin(true, false, newNd)\n\t\t\t}\n\t\t\treturn true\n\t\t},\n\t\tOnRemove: func(oldNd n.ModNode) bool {\n\t\t\tdoPinOrUnpin(false, true, oldNd)\n\t\t\treturn true\n\t\t},\n\t\t// OnMerge: func(newNd, oldNd n.ModNode) bool {\n\t\tOnMerge: func(nd n.ModNode, isGet bool, ndPinStats *vcs.PinStats) bool {\n\t\t\t// During merge we are acting on the same node\n\t\t\t// but we are modifying its hashes thus we would not be\n\t\t\t// able to get node pins by reusing node itself.\n\t\t\t// Our main goal is either get pin info from nd node\n\t\t\t// or to set it according to the previously obtained info.\n\t\t\tif isGet {\n\t\t\t\tisPinned, isExplicit, err := fs.pinner.IsNodePinned(nd)\n\t\t\t\tif err != nil {\n\t\t\t\t\tlog.Warnf(\n\t\t\t\t\t\t\"failed to check pin status of old node `%s` (%v)\",\n\t\t\t\t\t\tnd.Path(),\n\t\t\t\t\t\tnd.BackendHash(),\n\t\t\t\t\t)\n\n\t\t\t\t\t// better don't change something.\n\t\t\t\t\treturn false\n\t\t\t\t}\n\t\t\t\tndPinStats.Pinned = isPinned\n\t\t\t\tndPinStats.Explicit = isExplicit\n\n\t\t\t\treturn true\n\t\t\t}\n\t\t\t// If are not getting then we are setting\n\t\t\t// Pin new node with old pin state:\n\t\t\tif ndPinStats.Pinned {\n\t\t\t\t// Preserving the state\n\t\t\t\tdoPinOrUnpin(ndPinStats.Pinned, ndPinStats.Explicit, nd)\n\t\t\t} else {\n\t\t\t\t// if it was unpinned we are forcing it to be unppinned\n\t\t\t\tdoPinOrUnpin(false, true, nd)\n\t\t\t}\n\t\t\treturn true\n\t\t},\n\t\tOnConflict: func(src, dst n.ModNode) bool {\n\t\t\t// Don't need to do something,\n\t\t\t// conflict files will not get a pin by default.\n\t\t\treturn true\n\t\t},\n\t}, nil\n}\n\n// SyncOption is a option that can be passed to Sync.\ntype SyncOption func(cfg *vcs.SyncOptions)\n\n// SyncOptMessage sets the commit message that will be\n// given to MakeCommit() on a sync commit.\nfunc SyncOptMessage(msg string) SyncOption {\n\treturn func(cfg *vcs.SyncOptions) {\n\t\tcfg.Message = msg\n\t}\n}\n\n// SyncOptConflictStrategy overwrites the conflict strategy\n// (see also fs.sync.conflict_strategy which acts as default)\nfunc SyncOptConflictStrategy(strategy string) SyncOption {\n\treturn func(cfg *vcs.SyncOptions) {\n\t\tif strategy == \"\" {\n\t\t\treturn\n\t\t}\n\n\t\tcfg.ConflictStrategy = vcs.ConflictStrategyFromString(strategy)\n\t}\n}\n\n// SyncOptReadOnlyFolders allows you to set a set of folders\n// that will be protected from modifications by the sync.\nfunc SyncOptReadOnlyFolders(folders []string) SyncOption {\n\treturn func(cfg *vcs.SyncOptions) {\n\t\tif cfg.ReadOnlyFolders == nil {\n\t\t\tcfg.ReadOnlyFolders = make(map[string]bool)\n\t\t}\n\n\t\tfor _, folder := range folders {\n\t\t\tcfg.ReadOnlyFolders[folder] = true\n\t\t}\n\t}\n}\n\n// SyncOptConflictgStrategyPerFolder allows you to set a specific conflict\n// resolution strategy for specific folders. The key of the map is the folder,\n// the key is the conflict strategy name.\nfunc SyncOptConflictgStrategyPerFolder(strategies map[string]string) SyncOption {\n\treturn func(cfg *vcs.SyncOptions) {\n\t\tif cfg.ConflictStrategyPerFolder == nil {\n\t\t\tcfg.ConflictStrategyPerFolder = make(map[string]vcs.ConflictStrategy)\n\t\t}\n\n\t\tfor folder, strategy := range strategies {\n\t\t\tcs := vcs.ConflictStrategyFromString(strategy)\n\t\t\tif cs == vcs.ConflictStragetyUnknown {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tcfg.ConflictStrategyPerFolder[folder] = cs\n\t\t}\n\t}\n}\n\n// Sync will synchronize the state of two filesystems.\n// If one of filesystems have unstaged changes, they will be committted first.\n// If our filesystem was changed by Sync(), a new merge commit will also be created.\nfunc (fs *FS) Sync(remote *FS, options ...SyncOption) error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tif fs.readOnly {\n\t\treturn ErrReadOnly\n\t}\n\n\t// build default config from the defaults/base config:\n\tsyncCfg, err := fs.buildSyncCfg()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor _, option := range options {\n\t\toption(syncCfg)\n\t}\n\n\treturn vcs.Sync(remote.lkr, fs.lkr, syncCfg)\n}\n\n// MakeDiff will return a diff between `headRevOwn` and `headRevRemote`.\n// `remote` is the filesystem `headRevRemote` belongs to and may be the same as `fs`.\nfunc (fs *FS) MakeDiff(remote *FS, headRevOwn, headRevRemote string) (*Diff, error) {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tsrcHead, err := parseRev(remote.lkr, headRevRemote)\n\tif err != nil {\n\t\treturn nil, e.Wrapf(err, \"parse remote ref\")\n\t}\n\n\tdstHead, err := parseRev(fs.lkr, headRevOwn)\n\tif err != nil {\n\t\treturn nil, e.Wrapf(err, \"parse own ref\")\n\t}\n\n\tsyncCfg, err := fs.buildSyncCfg()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\trealDiff, err := vcs.MakeDiff(remote.lkr, fs.lkr, srcHead, dstHead, syncCfg)\n\tif err != nil {\n\t\treturn nil, e.Wrapf(err, \"make diff\")\n\t}\n\n\t// \"fake\" is the diff that we give to the outside.\n\t// Internally we have a bit more knowledge.\n\tfakeDiff := &Diff{}\n\n\t// Convert the simple slice parts:\n\tfor _, nd := range realDiff.Added {\n\t\tfakeDiff.Added = append(fakeDiff.Added, *fs.nodeToStat(nd))\n\t}\n\n\tfor _, nd := range realDiff.Ignored {\n\t\tfakeDiff.Ignored = append(fakeDiff.Ignored, *fs.nodeToStat(nd))\n\t}\n\n\tfor _, nd := range realDiff.Removed {\n\t\tfakeDiff.Removed = append(fakeDiff.Removed, *fs.nodeToStat(nd))\n\t}\n\n\tfor _, nd := range realDiff.Missing {\n\t\tfakeDiff.Missing = append(fakeDiff.Missing, *fs.nodeToStat(nd))\n\t}\n\n\t// And also convert the slightly more complex pairs:\n\tfor _, pair := range realDiff.Moved {\n\t\tfakeDiff.Moved = append(fakeDiff.Moved, DiffPair{\n\t\t\tSrc: *fs.nodeToStat(pair.Src),\n\t\t\tDst: *fs.nodeToStat(pair.Dst),\n\t\t})\n\t}\n\n\tfor _, pair := range realDiff.Merged {\n\t\tfakeDiff.Merged = append(fakeDiff.Merged, DiffPair{\n\t\t\tSrc: *fs.nodeToStat(pair.Src),\n\t\t\tDst: *fs.nodeToStat(pair.Dst),\n\t\t})\n\t}\n\n\tfor _, pair := range realDiff.Conflict {\n\t\tfakeDiff.Conflict = append(fakeDiff.Conflict, DiffPair{\n\t\t\tSrc: *fs.nodeToStat(pair.Src),\n\t\t\tDst: *fs.nodeToStat(pair.Dst),\n\t\t})\n\t}\n\n\treturn fakeDiff, nil\n}\n\nfunc (fs *FS) buildCommitHashToRefTable() (map[string][]string, error) {\n\tnames, err := fs.lkr.ListRefs()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\thashToRef := make(map[string][]string)\n\tfor _, name := range names {\n\t\tcmt, err := fs.lkr.ResolveRef(name)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif cmt != nil {\n\t\t\tkey := cmt.TreeHash().B58String()\n\t\t\thashToRef[key] = append(hashToRef[key], name)\n\t\t}\n\t}\n\n\treturn hashToRef, nil\n}\n\n// Log returns a list of commits starting with the staging commit until the\n// initial commit. For each commit, metadata is collected and fn is called.\n// The log starts at the revision pointed to by `head`.\n// If `head` is an empty string, \"curr\" is assumed.\nfunc (fs *FS) Log(head string, fn func(c *Commit) error) error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tvar (\n\t\theadCmt *n.Commit\n\t\terr error\n\t)\n\n\tif head == \"\" {\n\t\theadCmt, err = fs.lkr.Status()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t} else {\n\t\theadCmt, err = parseRev(fs.lkr, head)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\thashToRef, err := fs.buildCommitHashToRefTable()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn c.Log(fs.lkr, headCmt, func(cmt *n.Commit) error {\n\t\treturn fn(commitToExternal(cmt, hashToRef))\n\t})\n}\n\n// Reset restores the state of `path` to the state in `rev`.\nfunc (fs *FS) Reset(path, rev string) error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tif fs.readOnly {\n\t\treturn ErrReadOnly\n\t}\n\n\tif path == \"/\" || path == \"\" {\n\t\treturn fs.checkout(rev, false)\n\t}\n\n\tcmt, err := parseRev(fs.lkr, rev)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\toldNode, err := vcs.ResetNode(fs.lkr, cmt, path)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// The old node does not necessarily exist:\n\tif oldNode != nil {\n\t\tif err := fs.pinner.UnpinNode(oldNode, false); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\t// Cannot (un)pin non-existing file anymore.\n\tnewNode, err := fs.lkr.LookupNode(path)\n\tif ie.IsNoSuchFileError(err) {\n\t\treturn nil\n\t}\n\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn fs.pinner.PinNode(newNode, false)\n}\n\n// Checkout reverts all state to the commit referenced by `rev`.\n// If `force` is true a non-empty staging area will be overwritten.\nfunc (fs *FS) Checkout(rev string, force bool) error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\treturn fs.checkout(rev, force)\n}\n\nfunc (fs *FS) checkout(rev string, force bool) error {\n\tcmt, err := parseRev(fs.lkr, rev)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn fs.lkr.CheckoutCommit(cmt, force)\n}\n\n// Tag saves a human readable name for the revision pointed to by `rev`.\n// There are three pre-defined tags available:\n//\n// - HEAD: The last full commit.\n// - CURR: The current commit (== staging commit)\n// - INIT: the initial commit.\n//\n// The tagname is case-insensitive.\nfunc (fs *FS) Tag(rev, name string) error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tcmt, err := parseRev(fs.lkr, rev)\n\tif err != nil {\n\t\treturn e.Wrap(err, \"parse ref\")\n\t}\n\n\treturn fs.lkr.SaveRef(name, cmt)\n}\n\n// RemoveTag removes a previously created tag.\nfunc (fs *FS) RemoveTag(name string) error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\treturn fs.lkr.RemoveRef(name)\n}\n\n// FilesByContent returns all stat info for the content hashes referenced in\n// `contents`. The return value is a map with the content hash as key and a\n// StatInfo describing the exact file content.\nfunc (fs *FS) FilesByContent(contents []h.Hash) (map[string]StatInfo, error) {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tfiles, err := fs.lkr.FilesByContents(contents)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tinfos := make(map[string]StatInfo)\n\tfor content, file := range files {\n\t\tinfos[content] = *fs.nodeToStat(file)\n\t}\n\n\treturn infos, nil\n}\n\n// ScheduleGCRun runs GC run at the next possible time.\n// This method does not block until the run is finished.\nfunc (fs *FS) ScheduleGCRun() {\n\t// Putting a value into gcControl might block,\n\t// so better do it in the background.\n\tgo func() {\n\t\tfs.gcControl <- true\n\t}()\n}\n\nfunc (fs *FS) writeLastPatchIndex(index int64) error {\n\tfromIndexData := []byte(strconv.FormatInt(index, 10))\n\treturn fs.lkr.MetadataPut(\"fs.last-merge-index\", fromIndexData)\n}\n\nfunc (fs *FS) autoCommitStagedChanges(remoteName string) error {\n\thaveStagedChanges, err := fs.lkr.HaveStagedChanges()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Commit changes if there are any.\n\t// This is a little unfortunate implication on how the current\n\t// way of sending getting patches work. Creating a patch itself\n\t// works with a staging commit, but the versioning does not work\n\t// anymore then, since the same version might have a different\n\t// set of changes.\n\tif !haveStagedChanges {\n\t\treturn nil\n\t}\n\n\towner, err := fs.lkr.Owner()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tmsg := fmt.Sprintf(\"auto commit on metadata request from »%s«\", remoteName)\n\treturn fs.lkr.MakeCommit(owner, msg)\n\n}\n\n// MakePatch creates a binary patch with all file changes starting with\n// `fromRev`. Note that commit information is not exported, only individual\n// file and directory changes.\n//\n// The byte structured returned by this method may change at any point\n// and may not be relied upon.\n//\n// The `remoteName` is the name of the remote we're creating the patch for.\n// It's only used for display purpose in the commit message.\nfunc (fs *FS) MakePatch(fromRev string, folders []string, remoteName string) ([]byte, error) {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tif err := fs.autoCommitStagedChanges(remoteName); err != nil {\n\t\treturn nil, err\n\t}\n\n\tfrom, err := parseRev(fs.lkr, fromRev)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tpatch, err := vcs.MakePatch(fs.lkr, from, folders)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tmsg, err := patch.ToCapnp()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn msg.Marshal()\n}\n\n// MakePatches works like MakePatch but produces individual patches for commit.\n// This allows to persist the history to some extent.\nfunc (fs *FS) MakePatches(fromRev string, folders []string, remoteName string) ([]byte, error) {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tif err := fs.autoCommitStagedChanges(remoteName); err != nil {\n\t\treturn nil, err\n\t}\n\n\tfrom, err := parseRev(fs.lkr, fromRev)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tpatches, err := vcs.MakePatches(fs.lkr, from, folders)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tmsg, err := patches.ToCapnp()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn msg.Marshal()\n}\n\n// ApplyPatch reads the binary patch coming from MakePatch and tries to apply it.\nfunc (fs *FS) ApplyPatch(data []byte) error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tmsg, err := capnp.Unmarshal(data)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tpatch := &vcs.Patch{}\n\tif err := patch.FromCapnp(msg); err != nil {\n\t\treturn err\n\t}\n\n\treturn fs.applyPatches(vcs.Patches{patch})\n}\n\n// ApplyPatches reads the binary patch coming from MakePatches and tries to apply them.\nfunc (fs *FS) ApplyPatches(data []byte) error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tmsg, err := capnp.Unmarshal(data)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tpatches := &vcs.Patches{}\n\tif err := patches.FromCapnp(msg); err != nil {\n\t\treturn err\n\t}\n\n\treturn fs.applyPatches(*patches)\n}\n\nfunc (fs *FS) applyPatches(patches vcs.Patches) error {\n\towner, err := fs.lkr.Owner()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\thighestIndex := int64(-1)\n\tfor _, patch := range patches {\n\t\tif err := vcs.ApplyPatch(fs.lkr, patch); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif idx := patch.CurrIndex; highestIndex < idx {\n\t\t\thighestIndex = idx\n\t\t}\n\n\t\tcmtMsg := fmt.Sprintf(\"apply patch with %d changes\", len(patch.Changes))\n\t\tif err := fs.lkr.MakeCommit(owner, cmtMsg); err != nil {\n\t\t\tif err == ie.ErrNoChange {\n\t\t\t\t// Empty commits are totally possible.\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\treturn err\n\t\t}\n\t}\n\n\t// Remember what patch index we merged last.\n\t// This info can be read via LastPatchIndex() to determine\n\t// the next version to get from the remote.\n\tfromIndexData := []byte(strconv.FormatInt(highestIndex, 10))\n\treturn fs.lkr.MetadataPut(\"fs.last-merge-index\", fromIndexData)\n}\n\n// LastPatchIndex will return the current version of this filesystem\n// regarding patch state.\nfunc (fs *FS) LastPatchIndex() (int64, error) {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tfromIndexData, err := fs.lkr.MetadataGet(\"fs.last-merge-index\")\n\tif err != nil && err != db.ErrNoSuchKey {\n\t\treturn -1, err\n\t}\n\n\t// If we did not merge yet with anyone we have to\n\t// ask for a full fetch.\n\tif err == db.ErrNoSuchKey {\n\t\treturn 0, nil\n\t}\n\n\treturn strconv.ParseInt(string(fromIndexData), 10, 64)\n}\n\n// CommitInfo returns detailed info about a certain commit.\nfunc (fs *FS) CommitInfo(rev string) (*Commit, error) {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tcmt, err := parseRev(fs.lkr, rev)\n\tif cmt == nil || ie.IsErrNoSuchRef(err) {\n\t\treturn nil, nil\n\t}\n\n\thashToRef, err := fs.buildCommitHashToRefTable()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn commitToExternal(cmt, hashToRef), nil\n}\n\n// HaveStagedChanges returns true if there are changes that were not committed yet.\nfunc (fs *FS) HaveStagedChanges() (bool, error) {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\treturn fs.lkr.HaveStagedChanges()\n}\n\n// IsCached will return true when the file is cached locally.\nfunc (fs *FS) IsCached(path string) (bool, error) {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\tnd, err := fs.lkr.LookupNode(path)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\tif nd.Type() == n.NodeTypeDirectory && nd.NChildren() == 0 {\n\t\treturn true, nil\n\t}\n\n\ttotalCount := 0\n\tcachedCount := 0\n\terrNotCachedSentinel := errors.New(\"not cached found\")\n\n\terr = n.Walk(fs.lkr, nd, true, func(child n.Node) error {\n\t\tif child.Type() != n.NodeTypeFile {\n\t\t\treturn nil\n\t\t}\n\n\t\ttotalCount++\n\t\tisCached, err := fs.bk.IsCached(child.BackendHash())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif isCached {\n\t\t\t// Make sure that we do not count empty directories\n\t\t\t// as pinned nodes.\n\t\t\tcachedCount++\n\t\t} else {\n\t\t\t// Return a special error here to stop Walk() iterating.\n\t\t\t// One file is enough to stop IsCached() from being true.\n\t\t\treturn errNotCachedSentinel\n\t\t}\n\n\t\treturn nil\n\t})\n\n\tif err != nil && err != errNotCachedSentinel {\n\t\treturn false, err\n\t}\n\n\treturn cachedCount == totalCount, nil\n}\n\n// Hints returns the hint manager passed to NewFilesystem\nfunc (fs *FS) Hints() HintManager {\n\treturn fs.hintManager\n}\n" }, { "path": "catfs/fs_test.go", "content": "package catfs\n\nimport (\n\t\"archive/tar\"\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"sort\"\n\t\"testing\"\n\t\"time\"\n\n\tc \"github.com/sahib/brig/catfs/core\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\t\"github.com/sahib/brig/catfs/mio\"\n\t\"github.com/sahib/brig/catfs/mio/chunkbuf\"\n\t\"github.com/sahib/brig/catfs/mio/compress\"\n\t\"github.com/sahib/brig/catfs/mio/pagecache/mdcache\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\t\"github.com/sahib/brig/defaults\"\n\t\"github.com/sahib/brig/repo/hints\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\t\"github.com/sahib/brig/util/testutil\"\n\t\"github.com/sahib/config\"\n\tlog \"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc init() {\n\tlog.SetLevel(log.WarnLevel)\n}\n\nfunc withDummyFSReadOnly(t *testing.T, readOnly bool, fn func(fs *FS)) {\n\tbackend := NewMemFsBackend()\n\towner := \"alice\"\n\n\tdbPath, err := ioutil.TempDir(\"\", \"brig-fs-test\")\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to create temp dir: %v\", err)\n\t}\n\n\tdefer func() {\n\t\tif err := os.RemoveAll(dbPath); err != nil {\n\t\t\tt.Fatalf(\"Failed to clean up %s: %v\", dbPath, err)\n\t\t}\n\t}()\n\n\tcfg, err := config.Open(nil, defaults.Defaults, config.StrictnessPanic)\n\trequire.Nil(t, err)\n\n\tfsCfg := cfg.Section(\"fs\")\n\n\tmdc, err := mdcache.New(mdcache.Options{\n\t\tMaxMemoryUsage: 1024 * 1024,\n\t})\n\trequire.NoError(t, err)\n\n\tfs, err := NewFilesystem(\n\t\tbackend,\n\t\tdbPath,\n\t\towner,\n\t\treadOnly,\n\t\tfsCfg,\n\t\tnil,\n\t\tmdc,\n\t)\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to create filesystem: %v\", err)\n\t}\n\n\tfn(fs)\n\n\tif err := fs.Close(); err != nil {\n\t\tt.Fatalf(\"Failed to close filesystem: %v\", err)\n\t}\n}\n\nfunc withDummyFS(t *testing.T, fn func(fs *FS)) {\n\twithDummyFSReadOnly(t, false, fn)\n}\n\nfunc TestStat(t *testing.T) {\n\tt.Parallel()\n\n\twithDummyFS(t, func(fs *FS) {\n\t\t_, err := fs.Stat(\"/sub/x\")\n\t\trequire.True(t, ie.IsNoSuchFileError(err))\n\n\t\tc.MustMkdir(t, fs.lkr, \"/sub\")\n\t\tfile := c.MustTouch(t, fs.lkr, \"/sub/x\", 1)\n\n\t\tinfo, err := fs.Stat(\"/sub/x\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, info.Path, \"/sub/x\")\n\t\trequire.Equal(t, info.IsDir, false)\n\t\trequire.Equal(t, info.Size, uint64(0))\n\t\trequire.Equal(t, info.Inode, file.Inode())\n\t\trequire.Equal(t, info.TreeHash, file.TreeHash())\n\n\t\tdata := make([]byte, 42)\n\t\trequire.Nil(t, fs.Stage(\"/sub/x\", bytes.NewReader(data)))\n\n\t\tinfo, err = fs.Stat(\"/sub/x\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, info.Size, uint64(len(data)))\n\t\trequire.Equal(t, info.TreeHash, file.TreeHash())\n\n\t\tinfo, err = fs.Stat(\"/sub\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, info.Path, \"/sub\")\n\t\trequire.Equal(t, info.IsDir, true)\n\t\trequire.Equal(t, uint64(len(data)), info.Size)\n\t})\n}\n\nfunc TestLogAndTag(t *testing.T) {\n\tt.Parallel()\n\n\twithDummyFS(t, func(fs *FS) {\n\t\tcmts := []*n.Commit{}\n\t\tfor idx := 0; idx < 10; idx++ {\n\t\t\t_, cmt := c.MustTouchAndCommit(t, fs.lkr, \"/x\", byte(idx))\n\n\t\t\thash := cmt.TreeHash().B58String()\n\t\t\tif err := fs.Tag(hash, fmt.Sprintf(\"tag%d\", idx)); err != nil {\n\t\t\t\tt.Fatalf(\"Failed to tag %v: %v\", hash, err)\n\t\t\t}\n\n\t\t\tcmts = append(cmts, cmt)\n\t\t}\n\n\t\tstatus, err := fs.lkr.Status()\n\t\trequire.Nil(t, err)\n\n\t\tcmts = append(cmts, status)\n\n\t\tlog := []*Commit{}\n\t\trequire.Nil(t, fs.Log(\"\", func(c *Commit) error {\n\t\t\tlog = append(log, c)\n\t\t\treturn nil\n\t\t}))\n\n\t\tfor idx, entry := range log {\n\t\t\tridx := len(cmts) - idx - 1\n\t\t\tcmt := cmts[ridx]\n\t\t\trequire.Equal(t, entry.Hash, cmt.TreeHash())\n\n\t\t\tmsg := fmt.Sprintf(\"cmt %d\", ridx)\n\t\t\ttags := []string{fmt.Sprintf(\"tag%d\", ridx)}\n\n\t\t\t// 0 is status, 1 is head, 10 is initial\n\t\t\tswitch idx {\n\t\t\tcase 0:\n\t\t\t\ttags = []string{\"curr\"}\n\t\t\t\tmsg = \"\"\n\t\t\tcase 1:\n\t\t\t\ttags = append(tags, \"head\")\n\t\t\tcase 10:\n\t\t\t\ttags = append(tags, \"init\")\n\t\t\t}\n\n\t\t\tsort.Sort(sort.Reverse(sort.StringSlice(entry.Tags)))\n\t\t\trequire.EqualValues(t, tags, entry.Tags)\n\t\t\trequire.Equal(t, entry.Msg, msg)\n\t\t}\n\t})\n}\n\nvar TestKey = []byte(\"01234567890ABCDE01234567890ABCDE\")\n\nfunc TestCat(t *testing.T) {\n\tt.Parallel()\n\n\twithDummyFS(t, func(fs *FS) {\n\t\traw := []byte{1, 2, 3}\n\t\trinRaw := bytes.NewBuffer(raw)\n\n\t\trin, isRaw, err := mio.NewInStream(\n\t\t\trinRaw,\n\t\t\t\"\",\n\t\t\tTestKey,\n\t\t\thints.Default(),\n\t\t)\n\t\trequire.Nil(t, err)\n\n\t\tbackendHash, err := fs.bk.Add(rin)\n\t\trequire.Nil(t, err)\n\n\t\tcontentHash := h.TestDummy(t, 23)\n\n\t\t// Stage the file manually (without fs.Stage)\n\t\t_, err = c.Stage(\n\t\t\tfs.lkr,\n\t\t\t\"/x\",\n\t\t\tcontentHash,\n\t\t\tbackendHash,\n\t\t\tuint64(len(raw)),\n\t\t\tint64(len(raw)),\n\t\t\tTestKey,\n\t\t\ttime.Now(),\n\t\t\tisRaw,\n\t\t)\n\t\trequire.Nil(t, err)\n\n\t\t// Cat the file again:\n\t\tstream, err := fs.Cat(\"/x\")\n\t\trequire.Nil(t, err)\n\n\t\t// Check if the returned stream really contains 1,2,3\n\t\tresult := bytes.NewBuffer(nil)\n\t\t_, err = stream.WriteTo(result)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, result.Bytes(), raw)\n\t})\n}\n\nfunc TestStageBasic(t *testing.T) {\n\tt.Parallel()\n\n\tlog.SetLevel(log.DebugLevel)\n\ttcs := []int64{\n\t\t0,\n\t\t1,\n\t\t3,\n\t\t8 * 1024,\n\t\t64*1024 + 1,\n\t\t4 * 1024 * 1024,\n\t}\n\n\tfor _, size := range tcs {\n\t\tt.Run(fmt.Sprintf(\"size-%d\", size), func(t *testing.T) {\n\t\t\twithDummyFS(t, func(fs *FS) {\n\t\t\t\ttc := testutil.CreateDummyBuf(size)\n\t\t\t\tbuf := chunkbuf.NewChunkBuffer(tc)\n\t\t\t\trequire.NoError(t, fs.Stage(\"/x\", buf))\n\n\t\t\t\tstream, err := fs.Cat(\"/x\")\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tdata, err := ioutil.ReadAll(stream)\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\trequire.Equal(t, len(tc), len(data))\n\t\t\t\trequire.Equal(t, tc, data)\n\t\t\t\trequire.NoError(t, stream.Close())\n\n\t\t\t\tfile, err := fs.lkr.LookupFile(\"/x\")\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tkey := file.Key()\n\t\t\t\toldKey := make([]byte, len(key))\n\t\t\t\toldSize := file.Size()\n\t\t\t\tcopy(oldKey, key)\n\n\t\t\t\t// Also insert some more data to modify an existing file.\n\t\t\t\tnextData := []byte{6, 6, 6, 6, 6, 6}\n\t\t\t\trequire.NoError(t, fs.Stage(\"/x\", chunkbuf.NewChunkBuffer((nextData))))\n\t\t\t\tstream, err = fs.Cat(\"/x\")\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tdata, err = ioutil.ReadAll(stream)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.Equal(t, data, nextData)\n\t\t\t\trequire.NoError(t, stream.Close())\n\n\t\t\t\t// Check that the key did not change during modifying an existing file.\n\t\t\t\t// This is only true if both of the sizes are not equal to zero\n\t\t\t\t// Recall that 0 sized file has emptyFileEncryptionKey\n\t\t\t\tfile, err = fs.lkr.LookupFile(\"/x\")\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tif (oldSize != 0 && file.Size() != 0) || (oldSize == file.Size()) {\n\t\t\t\t\trequire.Equal(t, file.Key(), oldKey)\n\t\t\t\t} else {\n\t\t\t\t\trequire.NotEqual(t, file.Key(), oldKey)\n\t\t\t\t}\n\t\t\t})\n\t\t})\n\t}\n}\n\nfunc TestHistory(t *testing.T) {\n\tt.Parallel()\n\n\twithDummyFS(t, func(fs *FS) {\n\t\trequire.Nil(t, fs.MakeCommit(\"hello\"))\n\t\trequire.Nil(t, fs.Stage(\"/x\", chunkbuf.NewChunkBuffer([]byte{1})))\n\t\trequire.Nil(t, fs.MakeCommit(\"1\"))\n\t\trequire.Nil(t, fs.Stage(\"/x\", chunkbuf.NewChunkBuffer([]byte{2})))\n\t\trequire.Nil(t, fs.MakeCommit(\"2\"))\n\t\trequire.Nil(t, fs.Stage(\"/x\", chunkbuf.NewChunkBuffer([]byte{3})))\n\t\trequire.Nil(t, fs.MakeCommit(\"3\"))\n\n\t\thist, err := fs.History(\"/x\")\n\t\trequire.Nil(t, err)\n\n\t\tlog := []*Commit{}\n\t\trequire.Nil(t, fs.Log(\"\", func(c *Commit) error {\n\t\t\tlog = append(log, c)\n\t\t\treturn nil\n\t\t}))\n\n\t\tfor idx, entry := range hist {\n\t\t\trequire.Equal(t, entry.Path, \"/x\")\n\n\t\t\tchange := \"none\"\n\t\t\tswitch idx {\n\t\t\tcase 1, 2:\n\t\t\t\tchange = \"modified\"\n\t\t\tcase 3:\n\t\t\t\tchange = \"added\"\n\t\t\t}\n\n\t\t\trequire.Equal(t, entry.Change, change)\n\t\t\trequire.Equal(\n\t\t\t\tt,\n\t\t\t\tlog[idx].Hash.B58String(),\n\t\t\t\tentry.Head.Hash.B58String(),\n\t\t\t)\n\t\t}\n\t})\n}\n\nfunc mustReadPath(t *testing.T, fs *FS, path string) []byte {\n\tstream, err := fs.Cat(path)\n\trequire.Nil(t, err)\n\n\tdata, err := ioutil.ReadAll(stream)\n\trequire.Nil(t, err)\n\n\treturn data\n}\n\nfunc TestReset(t *testing.T) {\n\tt.Parallel()\n\n\twithDummyFS(t, func(fs *FS) {\n\t\trequire.Nil(t, fs.MakeCommit(\"hello\"))\n\n\t\trequire.Nil(t, fs.Stage(\"/x\", chunkbuf.NewChunkBuffer([]byte{1})))\n\t\trequire.Nil(t, fs.MakeCommit(\"1\"))\n\n\t\t// Modify on stage:\n\t\trequire.Nil(t, fs.Stage(\"/x\", chunkbuf.NewChunkBuffer([]byte{2})))\n\t\trequire.Nil(t, fs.Reset(\"/x\", \"HEAD\"))\n\n\t\tdata := mustReadPath(t, fs, \"/x\")\n\t\trequire.Equal(t, data[0], byte(1))\n\t\tif err := fs.MakeCommit(\"2\"); err != ie.ErrNoChange {\n\t\t\tt.Fatalf(\"Reset did clearly not reset stuff... (something changed)\")\n\t\t}\n\n\t\t// Remove the file and then reset it (like git checkout -- file)\n\t\trequire.Nil(t, fs.Remove(\"/x\"))\n\t\tif _, err := fs.Cat(\"/x\"); !ie.IsNoSuchFileError(err) {\n\t\t\tt.Fatalf(\"Something wrong with removed node\")\n\t\t}\n\n\t\t// Check if we can recover the delete:\n\t\trequire.Nil(t, fs.Reset(\"/x\", \"HEAD\"))\n\t\tdata = mustReadPath(t, fs, \"/x\")\n\t\trequire.Equal(t, data[0], byte(1))\n\n\t\t// Reset to something non-existing -> error.\n\t\trequire.NotNil(t, fs.Reset(\"/x\", \"DEADBEEF\"))\n\n\t\t// Reset to the very first commit - node did not exist back then.\n\t\trequire.Nil(t, fs.Reset(\"/x\", \"INIT\"))\n\n\t\t// Should not exist anymore currently.\n\t\t_, err := fs.Stat(\"/x\")\n\t\trequire.True(t, ie.IsNoSuchFileError(err))\n\t})\n}\n\nfunc TestCheckout(t *testing.T) {\n\tt.Parallel()\n\n\twithDummyFS(t, func(fs *FS) {\n\t\trequire.Nil(t, fs.MakeCommit(\"hello\"))\n\t\thello, err := fs.Head()\n\t\trequire.Nil(t, err)\n\n\t\trequire.Nil(t, fs.Touch(\"/x\"))\n\t\trequire.Nil(t, fs.Touch(\"/y\"))\n\t\trequire.Nil(t, fs.Touch(\"/z\"))\n\t\trequire.Nil(t, fs.Stage(\"/x\", bytes.NewReader([]byte{1, 2, 3})))\n\n\t\trequire.Nil(t, fs.Remove(\"/y\"))\n\t\trequire.Nil(t, fs.Move(\"/z\", \"/a\"))\n\n\t\trequire.Nil(t, fs.MakeCommit(\"world\"))\n\t\tworld, err := fs.Head()\n\t\trequire.Nil(t, err)\n\n\t\trequire.Nil(t, fs.Touch(\"/new\"))\n\t\trequire.Nil(t, fs.Stage(\"/x\", bytes.NewReader([]byte{4, 5, 6})))\n\n\t\terr = fs.Checkout(world, false)\n\t\trequire.Equal(t, err, ie.ErrStageNotEmpty)\n\n\t\terr = fs.Checkout(world, true)\n\t\trequire.Nil(t, err)\n\n\t\t_, err = fs.Stat(\"/new\")\n\t\trequire.True(t, ie.IsNoSuchFileError(err))\n\n\t\txStream, err := fs.Cat(\"/x\")\n\t\trequire.Nil(t, err)\n\t\tdata, err := ioutil.ReadAll(xStream)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, data, []byte{1, 2, 3})\n\n\t\terr = fs.Checkout(hello, true)\n\t\trequire.Nil(t, err)\n\n\t\t_, err = fs.Stat(\"/x\")\n\t\trequire.True(t, ie.IsNoSuchFileError(err))\n\t})\n}\n\nfunc TestExportImport(t *testing.T) {\n\tt.Parallel()\n\n\twithDummyFS(t, func(fs *FS) {\n\t\trequire.Nil(t, fs.MakeCommit(\"hello world\"))\n\n\t\t// Add a single file:\n\t\tbuf := chunkbuf.NewChunkBuffer([]byte{1, 2, 3})\n\t\trequire.Nil(t, fs.Stage(\"/x\", buf))\n\t\trequire.Nil(t, fs.MakeCommit(\"touchy touchy\"))\n\n\t\t// Stage something to see if this will also be exported\n\t\t// (it most defintely should)\n\t\tbuf = chunkbuf.NewChunkBuffer([]byte{3, 2, 1})\n\t\trequire.Nil(t, fs.Stage(\"/x\", buf))\n\n\t\tmem := &bytes.Buffer{}\n\t\trequire.Nil(t, fs.Export(mem))\n\n\t\t// Check if we can import all this data:\n\t\twithDummyFS(t, func(newFs *FS) {\n\t\t\trequire.Nil(t, fs.Import(mem))\n\n\t\t\tstream, err := fs.Cat(\"/x\")\n\t\t\trequire.Nil(t, err)\n\n\t\t\tdata, err := ioutil.ReadAll(stream)\n\t\t\trequire.Nil(t, err)\n\t\t\trequire.Equal(t, []byte{3, 2, 1}, data)\n\t\t})\n\t})\n}\n\nfunc TestSync(t *testing.T) {\n\tt.Parallel()\n\n\t// There are a lot more tests in vcs/*\n\t// This is only a test to see if the high-level api is working.\n\twithDummyFS(t, func(fsa *FS) {\n\t\trequire.Nil(t, fsa.MakeCommit(\"hello a\"))\n\t\twithDummyFS(t, func(fsb *FS) {\n\t\t\trequire.Nil(t, fsb.MakeCommit(\"hello b\"))\n\t\t\trequire.Nil(t, fsa.Sync(fsb))\n\n\t\t\trequire.Nil(t, fsb.Stage(\"/x\", bytes.NewReader([]byte{1})))\n\t\t\trequire.Nil(t, fsb.Stage(\"/y\", bytes.NewReader([]byte{2})))\n\t\t\trequire.Nil(t, fsb.Stage(\"/z\", bytes.NewReader([]byte{3})))\n\n\t\t\t// Actually sync the results:\n\t\t\trequire.Nil(t, fsa.Sync(fsb))\n\n\t\t\tinfo, err := fsa.Stat(\"/x\")\n\t\t\trequire.Nil(t, err)\n\t\t\trequire.Equal(t, info.Path, \"/x\")\n\n\t\t\tinfo, err = fsa.Stat(\"/y\")\n\t\t\trequire.Nil(t, err)\n\t\t\trequire.Equal(t, info.Path, \"/y\")\n\n\t\t\tinfo, err = fsa.Stat(\"/z\")\n\t\t\trequire.Nil(t, err)\n\t\t\trequire.Equal(t, info.Path, \"/z\")\n\t\t})\n\t})\n}\n\nfunc TestMakeDiff(t *testing.T) {\n\tt.Parallel()\n\n\t// There are a lot more tests in vcs/*\n\t// This is only a test for the high-level api.\n\twithDummyFS(t, func(fsa *FS) {\n\t\tfsaX := c.MustTouch(t, fsa.lkr, \"/x\", 1)\n\t\tfsaY := c.MustTouch(t, fsa.lkr, \"/y\", 2)\n\t\tfsaZ := c.MustTouch(t, fsa.lkr, \"/z\", 3)\n\n\t\trequire.Nil(t, fsa.MakeCommit(\"hello a\"))\n\t\twithDummyFS(t, func(fsb *FS) {\n\t\t\trequire.Nil(t, fsb.MakeCommit(\"hello b\"))\n\t\t\trequire.Nil(t, fsa.Sync(fsb))\n\n\t\t\tfsbX := c.MustTouch(t, fsb.lkr, \"/x\", 4)\n\t\t\tc.MustTouch(t, fsb.lkr, \"/y\", 5)\n\t\t\tfsbZ := c.MustTouch(t, fsb.lkr, \"/z\", 6)\n\t\t\tfsbA := c.MustTouch(t, fsb.lkr, \"/a\", 7)\n\n\t\t\trequire.Nil(t, fsb.MakeCommit(\"stuff\"))\n\t\t\trequire.Nil(t, fsb.Remove(\"/y\"))\n\t\t\trequire.Nil(t, fsb.MakeCommit(\"before diff\"))\n\n\t\t\t// Use the upwards notation:\n\t\t\tdiff, err := fsa.MakeDiff(fsb, \"head^^^\", \"curr\")\n\t\t\trequire.Nil(t, err)\n\n\t\t\trequire.Equal(t, []StatInfo{*fsb.nodeToStat(fsbA)}, diff.Added)\n\t\t\trequire.Equal(t, []StatInfo{*fsa.nodeToStat(fsaY)}, diff.Removed)\n\t\t\trequire.Equal(t, []DiffPair{{\n\t\t\t\tSrc: *fsb.nodeToStat(fsbX),\n\t\t\t\tDst: *fsa.nodeToStat(fsaX),\n\t\t\t}, {\n\t\t\t\tSrc: *fsb.nodeToStat(fsbZ),\n\t\t\t\tDst: *fsa.nodeToStat(fsaZ),\n\t\t\t}}, diff.Conflict)\n\t\t})\n\t})\n}\n\nfunc TestPin(t *testing.T) {\n\tt.Parallel()\n\n\twithDummyFS(t, func(fs *FS) {\n\t\t// NOTE: Both files have the same content.\n\t\trequire.Nil(t, fs.Stage(\"/x\", bytes.NewReader([]byte{1})))\n\t\trequire.Nil(t, fs.Stage(\"/y\", bytes.NewReader([]byte{1})))\n\n\t\trequire.Nil(t, fs.Unpin(\"/x\", \"curr\", true))\n\t\trequire.Nil(t, fs.Unpin(\"/y\", \"curr\", true))\n\n\t\tisPinned, isExplicit, err := fs.IsPinned(\"/x\")\n\t\trequire.Nil(t, err)\n\t\trequire.False(t, isPinned)\n\t\trequire.False(t, isExplicit)\n\n\t\trequire.Nil(t, fs.Pin(\"/x\", \"curr\", true))\n\n\t\tisPinned, isExplicit, err = fs.IsPinned(\"/x\")\n\t\trequire.Nil(t, err)\n\t\trequire.True(t, isPinned)\n\t\trequire.True(t, isExplicit)\n\n\t\tisPinned, isExplicit, err = fs.IsPinned(\"/\")\n\t\trequire.Nil(t, err)\n\t\trequire.False(t, isPinned)\n\t\trequire.False(t, isExplicit)\n\n\t\trequire.Nil(t, fs.Pin(\"/\", \"curr\", true))\n\n\t\tisPinned, isExplicit, err = fs.IsPinned(\"/\")\n\t\trequire.Nil(t, err)\n\t\trequire.True(t, isPinned)\n\t\trequire.True(t, isExplicit)\n\n\t\trequire.Nil(t, fs.Unpin(\"/\", \"curr\", true))\n\n\t\tisPinned, isExplicit, err = fs.IsPinned(\"/\")\n\t\trequire.Nil(t, err)\n\t\trequire.False(t, isPinned)\n\t\trequire.False(t, isExplicit)\n\n\t\tisPinned, isExplicit, err = fs.IsPinned(\"/x\")\n\t\trequire.Nil(t, err)\n\t\trequire.False(t, isPinned)\n\t\trequire.False(t, isExplicit)\n\t})\n}\n\nfunc TestMkdir(t *testing.T) {\n\tt.Parallel()\n\n\twithDummyFS(t, func(fs *FS) {\n\t\terr := fs.Mkdir(\"/a/b/c/d\", false)\n\t\trequire.True(t, ie.IsNoSuchFileError(err))\n\n\t\t_, err = fs.Stat(\"/a\")\n\t\trequire.True(t, ie.IsNoSuchFileError(err))\n\n\t\terr = fs.Mkdir(\"/a/b/c/d\", true)\n\t\trequire.Nil(t, err)\n\n\t\tinfo, err := fs.Stat(\"/a\")\n\t\trequire.Nil(t, err)\n\t\trequire.True(t, info.IsDir)\n\n\t\t// Check that it still works if the directory exists\n\t\terr = fs.Mkdir(\"/a/b/c/d\", false)\n\t\trequire.Nil(t, err)\n\n\t\terr = fs.Mkdir(\"/a/b/c/d\", true)\n\t\trequire.Nil(t, err)\n\n\t\terr = fs.Mkdir(\"/a/b/c\", false)\n\t\trequire.Nil(t, err)\n\t})\n}\n\nfunc TestMove(t *testing.T) {\n\tt.Parallel()\n\n\twithDummyFS(t, func(fs *FS) {\n\t\trequire.Nil(t, fs.Touch(\"/x\"))\n\t\trequire.Nil(t, fs.Move(\"/x\", \"/y\"))\n\n\t\t_, err := fs.Stat(\"/x\")\n\t\trequire.True(t, ie.IsNoSuchFileError(err))\n\n\t\tinfo, err := fs.Stat(\"/y\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, info.Path, \"/y\")\n\t\trequire.False(t, info.IsDir)\n\t})\n}\n\nfunc TestTouch(t *testing.T) {\n\tt.Parallel()\n\n\twithDummyFS(t, func(fs *FS) {\n\t\trequire.NoError(t, fs.Touch(\"/y\"))\n\t\tyInfo, err := fs.Stat(\"/y\")\n\t\trequire.NoError(t, err)\n\n\t\t// Check that the empty file has emptyFileEncryptionKey\n\t\trequire.Equal(t, yInfo.Key, emptyFileEncryptionKey())\n\n\t\trequire.NoError(t, fs.Touch(\"/x\"))\n\t\toldInfo, err := fs.Stat(\"/x\")\n\t\trequire.NoError(t, err)\n\n\t\t// Double Check that the empty file has emptyFileEncryptionKey\n\t\trequire.Equal(t, oldInfo.Key, emptyFileEncryptionKey())\n\n\t\t// Check that two empty files have same backend hash\n\t\trequire.Equal(t, oldInfo.BackendHash, yInfo.BackendHash)\n\n\t\t// Check that two empty files have same conternt hash\n\t\trequire.Equal(t, oldInfo.ContentHash, yInfo.ContentHash)\n\n\t\trequire.NoError(t, fs.Stage(\"/x\", bytes.NewReader([]byte{1, 2, 3})))\n\n\t\trequire.NoError(t, fs.Touch(\"/x\"))\n\t\tnewInfo, err := fs.Stat(\"/x\")\n\t\trequire.NoError(t, err)\n\n\t\t// Check that the non empty file encryption key is different from emptyFileEncryptionKey\n\t\trequire.NotEqual(t, newInfo.Key, emptyFileEncryptionKey())\n\n\t\t// Check that the timestamp advanced only.\n\t\trequire.True(t, oldInfo.ModTime.Before(newInfo.ModTime))\n\n\t\t// Also check that the content was not deleted:\n\t\tstream, err := fs.Cat(\"/x\")\n\t\trequire.NoError(t, err)\n\n\t\tdata, err := ioutil.ReadAll(stream)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, data, []byte{1, 2, 3})\n\n\t\trequire.NoError(t, stream.Close())\n\t})\n}\n\nfunc TestHead(t *testing.T) {\n\tt.Parallel()\n\n\twithDummyFS(t, func(fs *FS) {\n\t\t_, err := fs.Head()\n\n\t\trequire.True(t, ie.IsErrNoSuchRef(err))\n\t\trequire.Nil(t, fs.MakeCommit(\"init\"))\n\n\t\tref, err := fs.Head()\n\t\trequire.Nil(t, err)\n\n\t\theadCmt, err := fs.lkr.ResolveRef(\"head\")\n\t\trequire.Nil(t, err)\n\n\t\trequire.Equal(t, headCmt.TreeHash().B58String(), ref)\n\t})\n}\n\nfunc TestList(t *testing.T) {\n\tt.Parallel()\n\n\twithDummyFS(t, func(fs *FS) {\n\t\trequire.Nil(t, fs.Touch(\"/x\"))\n\t\trequire.Nil(t, fs.Mkdir(\"/1/2/3/\", true))\n\t\trequire.Nil(t, fs.Touch(\"/1/2/3/y\"))\n\n\t\tentries, err := fs.List(\"/1/2\", -1)\n\t\trequire.Nil(t, err)\n\n\t\trequire.Equal(t, len(entries), 3)\n\t\trequire.Equal(t, entries[0].Path, \"/1/2\")\n\t\trequire.Equal(t, entries[1].Path, \"/1/2/3\")\n\t\trequire.Equal(t, entries[2].Path, \"/1/2/3/y\")\n\n\t\tentries, err = fs.List(\"/\", 1)\n\t\trequire.Nil(t, err)\n\n\t\trequire.Equal(t, 2, len(entries))\n\t\trequire.Equal(t, entries[0].Path, \"/1\")\n\t\trequire.Equal(t, entries[1].Path, \"/x\")\n\n\t\tdir, err := fs.lkr.LookupDirectory(\"/1\")\n\t\trequire.Nil(t, err)\n\n\t\t// Check if ghosts are being treated as not existent:\n\t\tc.MustMove(t, fs.lkr, dir, \"/666\")\n\t\t_, err = fs.List(\"/1\", -1)\n\t\trequire.True(t, ie.IsNoSuchFileError(err))\n\t\t_, err = fs.List(\"/666\", -1)\n\t\trequire.Nil(t, err)\n\t})\n}\n\nfunc TestTag(t *testing.T) {\n\tt.Parallel()\n\n\twithDummyFS(t, func(fs *FS) {\n\t\trequire.Nil(t, fs.Touch(\"/x\"))\n\t\trequire.Nil(t, fs.MakeCommit(\"init\"))\n\n\t\thead, err := fs.Head()\n\t\trequire.Nil(t, err)\n\n\t\t// try with an abbreviated tag name.\n\t\trequire.Nil(t, fs.Tag(head[:10], \"xxx\"))\n\t\tcmt, err := fs.lkr.ResolveRef(\"xxx\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, cmt.(*n.Commit).Message(), \"init\")\n\n\t\trequire.Nil(t, fs.RemoveTag(\"xxx\"))\n\t\tcmt, err = fs.lkr.ResolveRef(\"xxx\")\n\t\trequire.Nil(t, cmt)\n\t\trequire.True(t, ie.IsErrNoSuchRef(err))\n\t})\n}\n\nfunc TestStageUnmodified(t *testing.T) {\n\tt.Parallel()\n\n\twithDummyFS(t, func(fs *FS) {\n\t\trequire.Nil(t, fs.Stage(\"/x\", bytes.NewReader([]byte{1})))\n\t\tinfoOld, err := fs.Stat(\"/x\")\n\t\trequire.Nil(t, err)\n\n\t\t// Just to be sure:\n\t\ttime.Sleep(50 * time.Millisecond)\n\n\t\trequire.Nil(t, fs.Stage(\"/x\", bytes.NewReader([]byte{1})))\n\t\tinfoNew, err := fs.Stat(\"/x\")\n\t\trequire.Nil(t, err)\n\n\t\trequire.Equal(t, infoOld.ModTime, infoNew.ModTime)\n\t})\n}\n\nfunc TestTruncate(t *testing.T) {\n\tt.Parallel()\n\n\twithDummyFS(t, func(fs *FS) {\n\t\tdata := testutil.CreateDummyBuf(1024)\n\t\trequire.Nil(t, fs.Stage(\"/x\", bytes.NewReader(data)))\n\n\t\tfor _, size := range []int{1025, 512, 1, 0, 1024} {\n\t\t\tt.Run(fmt.Sprintf(\"size-%d\", size), func(t *testing.T) {\n\t\t\t\trequire.Nil(t, fs.Truncate(\"/x\", uint64(size)))\n\t\t\t\t// clamp to 1024 for assertion:\n\t\t\t\treadSize := size\n\t\t\t\tif size > 1024 {\n\t\t\t\t\treadSize = 1024\n\t\t\t\t}\n\n\t\t\t\tstream, err := fs.Cat(\"/x\")\n\t\t\t\trequire.Nil(t, err)\n\t\t\t\treadData, err := ioutil.ReadAll(stream)\n\t\t\t\trequire.Nil(t, err)\n\t\t\t\trequire.Equal(t, len(readData), readSize)\n\t\t\t\trequire.Equal(t, readData, data[:readSize])\n\t\t\t})\n\t\t}\n\n\t\trequire.NotNil(t, fs.Truncate(\"/\", 0))\n\t})\n}\n\nfunc TestChangingCompressAlgos(t *testing.T) {\n\tt.Parallel()\n\n\twithDummyFS(t, func(fs *FS) {\n\t\t// Create a file which will not be compressed.\n\t\toldData := testutil.CreateDummyBuf(compress.HeaderSizeThreshold - 1)\n\t\trequire.Nil(t, fs.Stage(\"/a-text-file.go\", bytes.NewReader(oldData)))\n\n\t\t// Second run will use another compress algorithm, since we're\n\t\t// over the header size limit in the compression guesser.\n\t\tnewData := testutil.CreateDummyBuf(compress.HeaderSizeThreshold + 1)\n\t\trequire.Nil(t, fs.Stage(\"/a-text-file.go\", bytes.NewReader(newData)))\n\n\t\tstream, err := fs.Cat(\"/a-text-file.go\")\n\t\trequire.Nil(t, err)\n\n\t\tgotData, err := ioutil.ReadAll(stream)\n\t\trequire.Nil(t, err)\n\n\t\trequire.Equal(t, newData, gotData)\n\t})\n}\n\nfunc TestPatch(t *testing.T) {\n\twithDummyFS(t, func(srcFs *FS) {\n\t\twithDummyFS(t, func(dstFs *FS) {\n\t\t\trequire.Nil(t, srcFs.MakeCommit(\"init\"))\n\t\t\trequire.Nil(t, srcFs.Touch(\"/x\"))\n\t\t\trequire.Nil(t, srcFs.MakeCommit(\"added x\"))\n\n\t\t\tsrcIndex, err := srcFs.LastPatchIndex()\n\t\t\trequire.Nil(t, err)\n\t\t\trequire.Equal(t, int64(0), srcIndex)\n\n\t\t\tdstIndex, err := dstFs.LastPatchIndex()\n\t\t\trequire.Nil(t, err)\n\t\t\trequire.Equal(t, int64(0), dstIndex)\n\n\t\t\tpatch, err := srcFs.MakePatch(\"commit[0]\", nil, \"\")\n\t\t\trequire.Nil(t, err)\n\n\t\t\trequire.Nil(t, dstFs.ApplyPatch(patch))\n\t\t\tsrcX, err := srcFs.Stat(\"/x\")\n\t\t\trequire.Nil(t, err)\n\n\t\t\tsrcIndex, err = srcFs.LastPatchIndex()\n\t\t\trequire.Nil(t, err)\n\t\t\trequire.Equal(t, int64(0), srcIndex)\n\n\t\t\tdstX, err := dstFs.Stat(\"/x\")\n\t\t\trequire.Nil(t, err)\n\n\t\t\trequire.Equal(t, srcX.Path, dstX.Path)\n\t\t\trequire.Equal(t, srcX.Size, dstX.Size)\n\t\t\trequire.Equal(t, srcX.ContentHash, dstX.ContentHash)\n\t\t\trequire.Equal(t, srcX.BackendHash, dstX.BackendHash)\n\n\t\t\tdstIndex, err = dstFs.LastPatchIndex()\n\t\t\trequire.Nil(t, err)\n\t\t\trequire.Equal(t, int64(2), dstIndex)\n\t\t})\n\t})\n}\n\nfunc TestTar(t *testing.T) {\n\twithDummyFS(t, func(fs *FS) {\n\t\trequire.Nil(t, fs.Stage(\"/a/file.png\", bytes.NewReader([]byte(\"hello\"))))\n\t\trequire.Nil(t, fs.Stage(\"/b/file.jpg\", bytes.NewReader([]byte(\"world\"))))\n\t\trequire.Nil(t, fs.Stage(\"/c/file.gif\", bytes.NewReader([]byte(\"!\"))))\n\n\t\tbuf := &bytes.Buffer{}\n\t\trequire.Nil(t, fs.Tar(\"/\", buf, func(info *StatInfo) bool {\n\t\t\t// Exclude the /c directory:\n\t\t\treturn info.Path != \"/c\"\n\t\t}))\n\n\t\tr := tar.NewReader(buf)\n\t\tfor idx := 0; ; idx++ {\n\t\t\thdr, err := r.Next()\n\t\t\tif err == io.EOF {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\trequire.Nil(t, err)\n\n\t\t\tdata, err := ioutil.ReadAll(r)\n\t\t\tswitch idx {\n\t\t\tcase 0:\n\t\t\t\trequire.Equal(t, []byte(\"hello\"), data)\n\t\t\t\trequire.Equal(t, \"a/file.png\", hdr.Name)\n\t\t\t\trequire.Equal(t, int64(5), hdr.Size)\n\t\t\tcase 1:\n\t\t\t\trequire.Equal(t, []byte(\"world\"), data)\n\t\t\t\trequire.Equal(t, \"b/file.jpg\", hdr.Name)\n\t\t\t\trequire.Equal(t, int64(5), hdr.Size)\n\t\t\tdefault:\n\t\t\t\trequire.True(t, false, \"should not be reached\")\n\t\t\t}\n\t\t}\n\t})\n}\n\nfunc TestReadOnly(t *testing.T) {\n\twithDummyFSReadOnly(t, true, func(fs *FS) {\n\t\terr := fs.Stage(\"/x\", bytes.NewReader([]byte{1, 2, 3}))\n\t\trequire.Equal(t, ErrReadOnly, err)\n\t})\n}\n\nfunc TestDeletedNodesDirectory(t *testing.T) {\n\twithDummyFS(t, func(fs *FS) {\n\t\trequire.Nil(t, fs.Mkdir(\"/dir_a\", true))\n\t\trequire.Nil(t, fs.Mkdir(\"/dir_b\", true))\n\t\trequire.Nil(t, fs.MakeCommit(\"added\"))\n\n\t\trequire.Nil(t, fs.Remove(\"/dir_a\"))\n\t\trequire.Nil(t, fs.Move(\"/dir_b\", \"/dir_c\"))\n\t\trequire.Nil(t, fs.MakeCommit(\"{re,}move\"))\n\n\t\tdels, err := fs.DeletedNodes(\"/\")\n\t\trequire.Nil(t, err)\n\t\trequire.Len(t, dels, 1)\n\t\trequire.Equal(t, \"/dir_a\", dels[0].Path)\n\t\trequire.True(t, dels[0].IsDir)\n\t})\n}\n\nfunc TestDeletedNodesFile(t *testing.T) {\n\twithDummyFS(t, func(fs *FS) {\n\n\t\trequire.Nil(t, fs.Stage(\"/a\", bytes.NewReader([]byte(\"hello\"))))\n\t\trequire.Nil(t, fs.Stage(\"/b\", bytes.NewReader([]byte(\"world\"))))\n\t\trequire.Nil(t, fs.MakeCommit(\"added\"))\n\n\t\trequire.Nil(t, fs.Remove(\"/a\"))\n\t\trequire.Nil(t, fs.Move(\"/b\", \"/c\"))\n\t\trequire.Nil(t, fs.MakeCommit(\"{re,}move\"))\n\n\t\tdels, err := fs.DeletedNodes(\"/\")\n\t\trequire.Nil(t, err)\n\t\trequire.Len(t, dels, 1)\n\t\trequire.Equal(t, \"/a\", dels[0].Path)\n\t\trequire.False(t, dels[0].IsDir)\n\t})\n}\n\nfunc TestUndeleteFile(t *testing.T) {\n\twithDummyFS(t, func(fs *FS) {\n\t\trequire.Nil(t, fs.Stage(\"/a\", bytes.NewReader([]byte(\"hello\"))))\n\t\trequire.Nil(t, fs.Stage(\"/b\", bytes.NewReader([]byte(\"world\"))))\n\t\trequire.Nil(t, fs.MakeCommit(\"initial\"))\n\n\t\trequire.Nil(t, fs.Remove(\"/a\"))\n\t\trequire.Nil(t, fs.Move(\"/b\", \"/c\"))\n\t\trequire.Nil(t, fs.MakeCommit(\"{re,}move\"))\n\n\t\trequire.Nil(t, fs.Undelete(\"/a\"))\n\t\tinfo, err := fs.Stat(\"/a\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, \"/a\", info.Path)\n\t\trequire.False(t, info.IsDir)\n\n\t\tstream, err := fs.Cat(\"/a\")\n\t\trequire.Nil(t, err)\n\t\tdata, err := ioutil.ReadAll(stream)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, []byte(\"hello\"), data)\n\n\t\t// This file was moved -> Don't bring it back.\n\t\trequire.NotNil(t, fs.Undelete(\"/b\"))\n\t})\n}\n\nfunc TestUndeleteDirectory(t *testing.T) {\n\twithDummyFS(t, func(fs *FS) {\n\t\trequire.Nil(t, fs.Stage(\"/dir/a\", bytes.NewReader([]byte(\"hello\"))))\n\t\trequire.Nil(t, fs.Stage(\"/dir/sub/b\", bytes.NewReader([]byte(\"world\"))))\n\t\trequire.Nil(t, fs.Mkdir(\"/dir/empty\", true))\n\t\trequire.Nil(t, fs.MakeCommit(\"initial\"))\n\n\t\trequire.Nil(t, fs.Remove(\"/dir\"))\n\t\trequire.Nil(t, fs.MakeCommit(\"remove\"))\n\n\t\t_, err := fs.Stat(\"/dir/a\")\n\t\trequire.True(t, ie.IsNoSuchFileError(err))\n\n\t\trequire.Nil(t, fs.Undelete(\"/dir\"))\n\t\tinfo, err := fs.Stat(\"/dir\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, \"/dir\", info.Path)\n\t\trequire.True(t, info.IsDir)\n\n\t\tentries, err := fs.List(\"/dir\", -1)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, 5, len(entries))\n\n\t\tpaths := []string{}\n\t\tfor _, entry := range entries {\n\t\t\tpaths = append(paths, entry.Path)\n\t\t}\n\n\t\trequire.Equal(t, []string{\n\t\t\t\"/dir\",\n\t\t\t\"/dir/a\",\n\t\t\t\"/dir/empty\",\n\t\t\t\"/dir/sub\",\n\t\t\t\"/dir/sub/b\",\n\t\t}, paths)\n\t})\n}\n" }, { "path": "catfs/handle.go", "content": "package catfs\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"sync\"\n\n\t\"github.com/sahib/brig/catfs/mio\"\n\t\"github.com/sahib/brig/catfs/mio/pagecache\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n)\n\nvar (\n\t// ErrIsClosed is returned when an operation is performed on an already\n\t// closed file.\n\tErrIsClosed = errors.New(\"file handle is closed\")\n)\n\n// Handle is a emulation of a os.File handle, as returned by os.Open()\n// It supports the usual operations like open, read, write and seek.\n// Take note that the flushing operation currently is quite expensive.\ntype Handle struct {\n\tfs *FS\n\tfile *n.File\n\tlock sync.Mutex\n\tlayer *pagecache.Layer\n\tstream mio.Stream\n\twasModified bool\n\tisClosed bool\n\treadOnly bool\n}\n\nfunc newHandle(fs *FS, file *n.File, readOnly bool) *Handle {\n\treturn &Handle{\n\t\tfs: fs,\n\t\tfile: file,\n\t\treadOnly: readOnly,\n\t}\n}\n\nfunc (hdl *Handle) initStreamIfNeeded() error {\n\tif hdl.fs.pageCache == nil {\n\t\treturn errors.New(\"no page cache was initialized\")\n\t}\n\n\tif hdl.stream != nil {\n\t\treturn nil\n\t}\n\n\t// Initialize the stream lazily to avoid I/O on open()\n\trawStream, err := hdl.fs.bk.Cat(hdl.file.BackendHash())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Stack the mio stack on top:\n\thdl.stream, err = mio.NewOutStream(\n\t\trawStream,\n\t\thdl.file.IsRaw(),\n\t\thdl.file.Key(),\n\t)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\thdl.layer, err = pagecache.NewLayer(\n\t\thdl.stream,\n\t\thdl.fs.pageCache,\n\t\tint64(hdl.file.Inode()),\n\t\tint64(hdl.file.Size()),\n\t)\n\n\treturn err\n}\n\n// Read will try to fill `buf` as much as possible.\n// The seek pointer will be advanced by the number of bytes written.\n// Take care, `buf` might still have contents, even if io.EOF was returned.\nfunc (hdl *Handle) Read(buf []byte) (int, error) {\n\thdl.lock.Lock()\n\tdefer hdl.lock.Unlock()\n\n\tif hdl.isClosed {\n\t\treturn 0, ErrIsClosed\n\t}\n\n\tif err := hdl.initStreamIfNeeded(); err != nil {\n\t\treturn 0, err\n\t}\n\n\treturn hdl.layer.Read(buf)\n}\n\n// ReadAt reads from the overlay at `off` into `buf`.\nfunc (hdl *Handle) ReadAt(buf []byte, off int64) (int, error) {\n\thdl.lock.Lock()\n\tdefer hdl.lock.Unlock()\n\n\tif hdl.isClosed {\n\t\treturn 0, ErrIsClosed\n\t}\n\n\tif err := hdl.initStreamIfNeeded(); err != nil {\n\t\treturn 0, err\n\t}\n\n\treturn hdl.layer.ReadAt(buf, off)\n}\n\n// Write will write the contents of `buf` to the current position.\n// It will return the number of currently written bytes.\nfunc (hdl *Handle) Write(buf []byte) (int, error) {\n\thdl.lock.Lock()\n\tdefer hdl.lock.Unlock()\n\n\tif hdl.readOnly {\n\t\treturn 0, ErrReadOnly\n\t}\n\n\tif hdl.isClosed {\n\t\treturn 0, ErrIsClosed\n\t}\n\n\tif err := hdl.initStreamIfNeeded(); err != nil {\n\t\treturn 0, err\n\t}\n\n\thdl.wasModified = true\n\treturn hdl.layer.Write(buf)\n}\n\n// WriteAt writes data from `buf` at offset `off` counted from the start (0 offset).\n// Mimics `WriteAt` from `io` package https://golang.org/pkg/io/#WriterAt\nfunc (hdl *Handle) WriteAt(buf []byte, off int64) (n int, err error) {\n\thdl.lock.Lock()\n\tdefer hdl.lock.Unlock()\n\n\tif hdl.readOnly {\n\t\treturn 0, ErrReadOnly\n\t}\n\n\tif hdl.isClosed {\n\t\treturn 0, ErrIsClosed\n\t}\n\n\tif err := hdl.initStreamIfNeeded(); err != nil {\n\t\treturn 0, err\n\t}\n\n\thdl.wasModified = true\n\treturn hdl.layer.WriteAt(buf, off)\n}\n\n// Seek will jump to the `offset` relative to `whence`.\n// There next read and write operation will start from this point.\nfunc (hdl *Handle) Seek(offset int64, whence int) (int64, error) {\n\thdl.lock.Lock()\n\tdefer hdl.lock.Unlock()\n\n\tif hdl.isClosed {\n\t\treturn 0, ErrIsClosed\n\t}\n\n\tif err := hdl.initStreamIfNeeded(); err != nil {\n\t\treturn 0, err\n\t}\n\n\tn, err := hdl.layer.Seek(offset, whence)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\treturn n, nil\n}\n\n// Truncate truncates the file to a specific length.\nfunc (hdl *Handle) Truncate(size uint64) error {\n\thdl.lock.Lock()\n\tdefer hdl.lock.Unlock()\n\n\tif hdl.readOnly {\n\t\treturn ErrReadOnly\n\t}\n\n\tif hdl.isClosed {\n\t\treturn ErrIsClosed\n\t}\n\n\tif err := hdl.initStreamIfNeeded(); err != nil {\n\t\treturn err\n\t}\n\n\thdl.fs.mu.Lock()\n\thdl.file.SetSize(size)\n\thdl.fs.mu.Unlock()\n\n\thdl.layer.Truncate(int64(size))\n\treturn nil\n}\n\n// unlocked version of Flush()\nfunc (hdl *Handle) flush() error {\n\t// flush unsets the layer, so we don't flush twice.\n\tif hdl.layer == nil {\n\t\treturn nil\n\t}\n\n\t// No need to flush anything if no write calls happened.\n\tif !hdl.wasModified {\n\t\treturn nil\n\t}\n\n\t// Make sure that hdl.layer is unset in any case.\n\t// but only do that in case of real writes.\n\tdefer func() {\n\t\thdl.layer = nil\n\t\thdl.stream = nil\n\t\thdl.wasModified = false\n\t}()\n\n\t// Jump back to the beginning of the file, since fs.Stage()\n\t// should read all content starting from there.\n\tn, err := hdl.layer.Seek(0, io.SeekStart)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif n != 0 {\n\t\treturn fmt.Errorf(\"seek offset is not 0\")\n\t}\n\n\tpath := hdl.file.Path()\n\tif err := hdl.fs.Stage(path, hdl.layer); err != nil {\n\t\treturn err\n\t}\n\n\treturn hdl.layer.Close()\n}\n\n// Flush makes sure to write the current state to the backend.\n// Please remember that this method is currently pretty expensive.\nfunc (hdl *Handle) Flush() error {\n\thdl.lock.Lock()\n\tdefer hdl.lock.Unlock()\n\n\tif hdl.readOnly {\n\t\treturn ErrReadOnly\n\t}\n\n\tif hdl.isClosed {\n\t\treturn ErrIsClosed\n\t}\n\n\treturn hdl.flush()\n}\n\n// Close will finalize the file. It should not be used after.\n// This will call flush if it did not happen yet.\nfunc (hdl *Handle) Close() error {\n\thdl.lock.Lock()\n\tdefer hdl.lock.Unlock()\n\n\tif hdl.isClosed {\n\t\treturn ErrIsClosed\n\t}\n\n\thdl.isClosed = true\n\treturn hdl.flush()\n}\n\n// Path returns the absolute path of the file.\nfunc (hdl *Handle) Path() string {\n\thdl.lock.Lock()\n\tdefer hdl.lock.Unlock()\n\n\treturn hdl.file.Path()\n}\n" }, { "path": "catfs/handle_test.go", "content": "package catfs\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"testing\"\n\n\t\"github.com/sahib/brig/catfs/mio/compress\"\n\t\"github.com/sahib/brig/util/testutil\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestOpenRead(t *testing.T) {\n\twithDummyFS(t, func(fs *FS) {\n\t\trawData := []byte{1, 2, 3}\n\t\trequire.Nil(t, fs.Stage(\"/x\", bytes.NewReader(rawData)))\n\n\t\tfd, err := fs.Open(\"/x\")\n\t\trequire.Nil(t, err)\n\n\t\tdata, err := ioutil.ReadAll(fd)\n\t\trequire.Nil(t, err)\n\n\t\trequire.Equal(t, data, rawData)\n\t\trequire.Nil(t, fd.Close())\n\t})\n}\n\nfunc TestOpenWrite(t *testing.T) {\n\twithDummyFS(t, func(fs *FS) {\n\t\trawData := []byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}\n\t\trequire.Nil(t, fs.Stage(\"/x\", bytes.NewReader(rawData)))\n\n\t\tfd, err := fs.Open(\"/x\")\n\t\trequire.Nil(t, err)\n\n\t\tn, err := fd.Write([]byte{3, 2, 1})\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, n, 3)\n\n\t\tdata, err := ioutil.ReadAll(fd)\n\t\trequire.Nil(t, err)\n\t\t// require.Equal(t, rawData[3:], data)\n\t\trequire.Equal(t, []byte{3, 2, 1, 4, 5, 6, 7, 8, 9, 10}, data)\n\n\t\t// Check that we can also seek back to start after reading to the end.\n\t\t// (and also check if the write overlay actually did work)\n\t\tpos, err := fd.Seek(0, io.SeekStart)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, pos, int64(0))\n\n\t\tdata, err = ioutil.ReadAll(fd)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, []byte{3, 2, 1, 4, 5, 6, 7, 8, 9, 10}, data)\n\t\trequire.Nil(t, fd.Close())\n\t})\n}\n\nfunc TestOpenTruncate(t *testing.T) {\n\trawData := []byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}\n\n\tfor idx := 0; idx < len(rawData)+5; idx++ {\n\t\tt.Run(fmt.Sprintf(\"truncate_%d\", idx), func(t *testing.T) {\n\t\t\twithDummyFS(t, func(fs *FS) {\n\t\t\t\trequire.Nil(t, fs.Stage(\"/x\", bytes.NewReader(rawData)))\n\n\t\t\t\tfd, err := fs.Open(\"/x\")\n\t\t\t\trequire.Nil(t, err)\n\n\t\t\t\trequire.Nil(t, fd.Truncate(uint64(idx)))\n\n\t\t\t\tdata, err := ioutil.ReadAll(fd)\n\t\t\t\trequire.Nil(t, err)\n\n\t\t\t\t// cap rawData index:\n\t\t\t\trawIdx := idx\n\t\t\t\tif idx >= len(rawData) {\n\t\t\t\t\trawIdx = len(rawData)\n\t\t\t\t}\n\n\t\t\t\trequire.Equal(t, rawData[:rawIdx], data)\n\t\t\t\trequire.Nil(t, fd.Close())\n\n\t\t\t\t// Check if the result was really written:\n\t\t\t\tstream, err := fs.Cat(\"/x\")\n\t\t\t\trequire.Nil(t, err)\n\n\t\t\t\tpersistentData, err := ioutil.ReadAll(stream)\n\t\t\t\trequire.Nil(t, err)\n\t\t\t\trequire.Equal(t, rawData[:rawIdx], persistentData)\n\t\t\t})\n\t\t})\n\t}\n}\n\nfunc TestOpenOpAfterClose(t *testing.T) {\n\twithDummyFS(t, func(fs *FS) {\n\t\trawData := []byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}\n\t\trequire.Nil(t, fs.Stage(\"/x\", bytes.NewReader(rawData)))\n\n\t\tfd, err := fs.Open(\"/x\")\n\t\trequire.Nil(t, err)\n\n\t\trequire.Nil(t, fd.Close())\n\n\t\t_, err = ioutil.ReadAll(fd)\n\t\trequire.Equal(t, err, ErrIsClosed)\n\t})\n}\n\n// TODO: More tests. This still feels buggy.\n// Cases needed for:\n// - 0, SEEK_END\n// - 9, SEEK_SET\n// - ...\nfunc TestOpenExtend(t *testing.T) {\n\tt.Run(\"start-10\", func(t *testing.T) {\n\t\ttestOpenExtend(t, 10, io.SeekStart)\n\t})\n\tt.Run(\"curr-10\", func(t *testing.T) {\n\t\ttestOpenExtend(t, 10, io.SeekCurrent)\n\t})\n\tt.Run(\"end-0\", func(t *testing.T) {\n\t\ttestOpenExtend(t, 0, io.SeekEnd)\n\t})\n}\n\nfunc testOpenExtend(t *testing.T, pos int64, whence int) {\n\twithDummyFS(t, func(fs *FS) {\n\t\trawData := []byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}\n\t\trequire.Nil(t, fs.Stage(\"/x\", bytes.NewReader(rawData)))\n\n\t\tfd, err := fs.Open(\"/x\")\n\t\trequire.Nil(t, err)\n\n\t\tpos, err := fd.Seek(pos, whence)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, pos, int64(pos))\n\n\t\tn, err := fd.Write([]byte{11, 12, 13})\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, n, 3)\n\n\t\trequire.Nil(t, fd.Close())\n\n\t\tstream, err := fs.Cat(\"/x\")\n\t\trequire.Nil(t, err)\n\n\t\tpostData, err := ioutil.ReadAll(stream)\n\t\trequire.Nil(t, err)\n\n\t\texpected := make([]byte, 13)\n\t\tcopy(expected, rawData)\n\t\tcopy(expected[10:], []byte{11, 12, 13})\n\t\trequire.Equal(t, expected, postData)\n\t})\n}\n\n// Read data from the handle like fuse would:\n// Seek to an offset, read a chunk and then advance to next block.\n// block size and file size may var heavily here.\nfunc TestHandleFuseLikeRead(t *testing.T) {\n\ttcs := []struct {\n\t\tfileSize int\n\t\tblockSize int\n\t}{\n\t\t{2048, 400},\n\t}\n\n\tfor _, tc := range tcs {\n\n\t\ttestHandleFuseLikeRead(t, tc.fileSize, tc.blockSize)\n\t}\n}\n\nfunc testHandleFuseLikeRead(t *testing.T, fileSize, blockSize int) {\n\t// fuse reads data always with a prior seek.\n\t// try to emulate this behaviour here.\n\twithDummyFS(t, func(fs *FS) {\n\t\trawData := testutil.CreateDummyBuf(int64(fileSize))\n\t\trequire.Nil(t, fs.Stage(\"/x\", bytes.NewReader(rawData)))\n\n\t\tfd, err := fs.Open(\"/x\")\n\t\trequire.Nil(t, err)\n\n\t\tleft := len(rawData)\n\t\tfor left > 0 {\n\t\t\ttoRead := blockSize\n\t\t\tif left < blockSize {\n\t\t\t\ttoRead = left\n\t\t\t}\n\n\t\t\toffset := len(rawData) - left\n\t\t\tbuf := make([]byte, toRead)\n\t\t\tif _, err = fd.Seek(int64(offset), io.SeekStart); err != nil {\n\t\t\t\tt.Fatalf(\"Seek to %d failed\", offset)\n\t\t\t}\n\n\t\t\tn, err := fd.Read(buf)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"Read failed: %v\", err)\n\t\t\t}\n\n\t\t\tif n != toRead {\n\t\t\t\tt.Fatalf(\"Handle read less than expected (wanted %d, got %d)\", toRead, n)\n\t\t\t}\n\n\t\t\tif !bytes.Equal(buf, rawData[offset:offset+toRead]) {\n\t\t\t\tt.Fatalf(\"Block [%d:%d] differs from raw data\", offset, offset+toRead)\n\t\t\t}\n\n\t\t\tleft -= blockSize\n\t\t}\n\n\t\trequire.Nil(t, fd.Close())\n\t})\n}\n\nfunc TestHandleChangeCompression(t *testing.T) {\n\twithDummyFS(t, func(fs *FS) {\n\t\t// Create a file which will not be compressed.\n\t\tsize := int64(compress.HeaderSizeThreshold - 1)\n\t\toldData := testutil.CreateDummyBuf(size)\n\t\trequire.Nil(t, fs.Mkdir(\"/sub\", false))\n\t\trequire.Nil(t, fs.Stage(\"/sub/a-text-file.go\", bytes.NewReader(oldData)))\n\n\t\t// Second run will use another compress algorithm, since we're\n\t\t// over the header size limit in the compression guesser.\n\t\tfd, err := fs.Open(\"/sub/a-text-file.go\")\n\t\trequire.Nil(t, err)\n\n\t\t// \"echo(1)\" does a flush after open (for whatever reason)\n\t\trequire.Nil(t, fd.Flush())\n\n\t\toffset, err := fd.Seek(size, io.SeekStart)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, offset, size)\n\n\t\texpectedData := []byte(\"xxxxx\")\n\t\tn, err := fd.Write(expectedData)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, n, len(expectedData))\n\t\trequire.Nil(t, fd.Flush())\n\t\trequire.Nil(t, fd.Close())\n\n\t\tstream, err := fs.Cat(\"/sub/a-text-file.go\")\n\t\trequire.Nil(t, err)\n\n\t\tgotData, err := ioutil.ReadAll(stream)\n\t\trequire.Nil(t, err)\n\n\t\texpectData := append(oldData, expectedData...)\n\t\trequire.Equal(t, expectData, gotData)\n\t})\n}\n" }, { "path": "catfs/mio/chunkbuf/chunkbuf.go", "content": "package chunkbuf\n\nimport (\n\t\"io\"\n\n\t\"github.com/sahib/brig/util\"\n)\n\n// ChunkBuffer represents a custom buffer struct with Read/Write and Seek support.\ntype ChunkBuffer struct {\n\tbuf []byte\n\treadOff int64\n\twriteOff int64\n\tsize int64\n}\n\nconst (\n\tmaxChunkSize = 64 * 1024\n)\n\nfunc (c *ChunkBuffer) Write(p []byte) (int, error) {\n\tn := copy(c.buf[c.writeOff:c.size], p)\n\tc.writeOff += int64(n)\n\tc.size = util.Max64(c.size, c.writeOff)\n\treturn n, nil\n}\n\n// Reset resets the buffer like bytes.Buffer\nfunc (c *ChunkBuffer) Reset(data []byte) {\n\tc.readOff = 0\n\tc.writeOff = 0\n\tc.size = int64(len(data))\n\tc.buf = data\n}\n\n// Len tells you the current size of the buffer contents\nfunc (c *ChunkBuffer) Len() int {\n\treturn int(c.size - c.readOff)\n}\n\nfunc (c *ChunkBuffer) Read(p []byte) (int, error) {\n\tn := copy(p, c.buf[c.readOff:c.size])\n\tc.readOff += int64(n)\n\tif n < len(p) {\n\t\treturn n, io.EOF\n\t}\n\n\treturn n, nil\n}\n\n// Seek implements io.Seeker\nfunc (c *ChunkBuffer) Seek(offset int64, whence int) (int64, error) {\n\tswitch whence {\n\tcase io.SeekCurrent:\n\t\tc.readOff += offset\n\tcase io.SeekEnd:\n\t\tc.readOff = c.size + offset\n\tcase io.SeekStart:\n\t\tc.readOff = offset\n\t}\n\tc.readOff = util.Min64(c.readOff, c.size)\n\tc.writeOff = c.readOff\n\treturn c.readOff, nil\n}\n\n// Close is a no-op only existing to fulfill io.Closer\nfunc (c *ChunkBuffer) Close() error {\n\treturn nil\n}\n\n// WriteTo implements the io.WriterTo interface\nfunc (c *ChunkBuffer) WriteTo(w io.Writer) (int64, error) {\n\tn, err := w.Write(c.buf[c.readOff:])\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\tc.readOff += int64(n)\n\treturn int64(n), nil\n}\n\n// NewChunkBuffer returns a ChunkBuffer with the given data. if data is nil a\n// ChunkBuffer with 64k is returned.\n// Note that chunkbuf will take over ownership over the buf.\nfunc NewChunkBuffer(data []byte) *ChunkBuffer {\n\tif data == nil {\n\t\tdata = make([]byte, maxChunkSize)\n\t}\n\n\treturn &ChunkBuffer{buf: data, size: int64(len(data))}\n}\n" }, { "path": "catfs/mio/chunkbuf/chunkbuf_test.go", "content": "package chunkbuf\n\nimport (\n\t\"bytes\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"testing\"\n\n\t\"github.com/sahib/brig/util/testutil\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestChunkBufBasic(t *testing.T) {\n\tdata := testutil.CreateDummyBuf(1024)\n\tbuf := NewChunkBuffer(data)\n\n\tcopiedData, err := ioutil.ReadAll(buf)\n\trequire.Nil(t, err)\n\trequire.Equal(t, data, copiedData)\n}\n\nfunc TestChunkBufEOF(t *testing.T) {\n\tdata := testutil.CreateDummyBuf(1024)\n\tbuf := NewChunkBuffer(data)\n\n\tcache := make([]byte, 2048)\n\tn, err := buf.Read(cache)\n\trequire.True(t, err == io.EOF)\n\trequire.Equal(t, n, 1024)\n\trequire.Nil(t, buf.Close())\n}\n\nfunc TestChunkBufWriteTo(t *testing.T) {\n\tdata := testutil.CreateDummyBuf(1024)\n\tbuf := NewChunkBuffer(data)\n\n\tstdBuf := &bytes.Buffer{}\n\tn, err := buf.WriteTo(stdBuf)\n\trequire.Nil(t, err)\n\trequire.Equal(t, int64(n), int64(1024))\n\trequire.Equal(t, data, stdBuf.Bytes())\n}\n\nfunc TestChunkBufSeek(t *testing.T) {\n\tdata := testutil.CreateDummyBuf(1024)\n\tbuf := NewChunkBuffer(data)\n\n\tvar err error\n\tvar n int\n\n\tcache := make([]byte, 128)\n\tn, err = buf.Read(cache)\n\trequire.Nil(t, err)\n\trequire.Equal(t, n, 128)\n\trequire.Equal(t, cache[:n], data[:n])\n\n\tjumpedTo, err := buf.Seek(256, io.SeekStart)\n\trequire.Nil(t, err)\n\trequire.Equal(t, int64(jumpedTo), int64(256))\n\n\tcache = make([]byte, 128)\n\tn, err = buf.Read(cache)\n\trequire.Nil(t, err)\n\trequire.Equal(t, n, 128)\n\trequire.Equal(t, cache[:n], data[256:n+256])\n\n\t// read advanced by 128, add 128 to go to 512\n\tjumpedTo, err = buf.Seek(128, io.SeekCurrent)\n\trequire.Nil(t, err)\n\trequire.Equal(t, int64(jumpedTo), int64(512))\n\n\tcache = make([]byte, 128)\n\tn, err = buf.Read(cache)\n\trequire.Nil(t, err)\n\trequire.Equal(t, n, 128)\n\trequire.Equal(t, cache[:n], data[512:n+512])\n\n\t// read advanced by 128, add 128 to go to 512\n\tjumpedTo, err = buf.Seek(-128, io.SeekEnd)\n\trequire.Nil(t, err)\n\trequire.Equal(t, int64(jumpedTo), int64(896))\n\n\tcache = make([]byte, 128)\n\tn, err = buf.Read(cache)\n\trequire.Nil(t, err)\n\trequire.Equal(t, n, 128)\n\trequire.Equal(t, cache[:n], data[896:n+896])\n}\n\nfunc TestChunkBufWrite(t *testing.T) {\n\tdata := testutil.CreateDummyBuf(1024)\n\tref := testutil.CreateDummyBuf(1024)\n\tbuf := NewChunkBuffer(data)\n\n\tref[0] = 1\n\tref[1] = 2\n\tref[2] = 3\n\n\tn, err := buf.Write([]byte{1, 2, 3})\n\trequire.Nil(t, err)\n\trequire.Equal(t, n, 3)\n\n\tjumpedTo, err := buf.Seek(-1, io.SeekEnd)\n\trequire.Nil(t, err)\n\trequire.Equal(t, int64(jumpedTo), int64(1023))\n\n\tref[1023] = 255\n\n\tn, err = buf.Write([]byte{255, 255, 255})\n\trequire.Nil(t, err)\n\trequire.Equal(t, n, 1)\n\n\tjumpedTo, err = buf.Seek(0, io.SeekStart)\n\trequire.Nil(t, err)\n\trequire.Equal(t, int64(jumpedTo), int64(0))\n\n\tstdBuf := &bytes.Buffer{}\n\tnWriteTo, err := buf.WriteTo(stdBuf)\n\trequire.Nil(t, err)\n\trequire.Equal(t, nWriteTo, int64(1024))\n\n\trequire.Equal(t, stdBuf.Bytes(), ref)\n}\n" }, { "path": "catfs/mio/compress/algorithm.go", "content": "package compress\n\nimport (\n\t\"errors\"\n\n\t\"github.com/golang/snappy\"\n\t\"github.com/klauspost/compress/zstd\"\n\t\"github.com/pierrec/lz4/v4\"\n)\n\nvar (\n\t// ErrBadAlgo is returned on a unsupported/unknown algorithm.\n\tErrBadAlgo = errors.New(\"invalid algorithm type\")\n)\n\nconst (\n\t// AlgoUnknown represents an unknown algorithm.\n\t// When trying to use it an error will occur.\n\tAlgoUnknown = AlgorithmType(iota)\n\n\t// AlgoSnappy represents the snappy compression algorithm:\n\t// https://en.wikipedia.org/wiki/Snappy_(software)\n\tAlgoSnappy\n\n\t//AlgoLZ4 represents the lz4 compression algorithm:\n\t// https://en.wikipedia.org/wiki/LZ4_(compression_algorithm)\n\tAlgoLZ4\n\n\t// AlgoZstd represents the zstd compression algorithm:\n\t// https://de.wikipedia.org/wiki/Zstandard\n\tAlgoZstd\n)\n\n// AlgorithmType user defined type to store the algorithm type.\ntype AlgorithmType byte\n\n// IsValid returns true if `at` is a valid algorithm type.\nfunc (at AlgorithmType) IsValid() bool {\n\tswitch at {\n\tcase AlgoSnappy, AlgoLZ4, AlgoZstd:\n\t\treturn true\n\t}\n\n\treturn false\n}\n\nfunc (at AlgorithmType) String() string {\n\tname, ok := algoToString[at]\n\tif !ok {\n\t\treturn \"unknown\"\n\t}\n\n\treturn name\n}\n\n// Algorithm is the common interface for all supported algorithms.\ntype Algorithm interface {\n\t// Encode should encode `src` into the buffer provided by `dst`.\n\t// It should return a sub-slice of `dst`. `dst` should be big\n\t// enough to hold `src`. Use MaxEncodeBufferSize() to be sure.\n\tEncode(dst, src []byte) ([]byte, error)\n\n\t// Decode decodes the data in `src` to `dst`, returning a subslice\n\t// of `dst` to indicate the actual size.\n\tDecode(dst, src []byte) ([]byte, error)\n\n\t// MaxEncodeBufferSize should return the maximum size an encoded\n\t// (i.e. compressed) buffer of input size maxChunkSize may have.\n\t// This will be bigger than maxChunkSize since random data will\n\t// be inflated by almost all algorithms.\n\tMaxEncodeBufferSize() int\n}\n\ntype snappyAlgo struct{}\n\ntype lz4Algo struct {\n\tcompressor *lz4.Compressor\n}\n\ntype zstdAlgo struct{}\n\nvar (\n\tzstdWriter *zstd.Encoder\n\tzstdReader *zstd.Decoder\n)\n\nfunc init() {\n\tvar err error\n\n\t// NOTE: zstd package allows us to use the same writer and reader\n\t// stateless if we just use block encoding/decoding.\n\t// This saves us some extra allocations.\n\n\t// TODO: configure compression level?\n\tzstdWriter, err = zstd.NewWriter(\n\t\tnil,\n\t\tzstd.WithEncoderLevel(zstd.SpeedDefault),\n\t)\n\n\tif err != nil {\n\t\t// configuring the writer wrong is a programmer error.\n\t\tpanic(err)\n\t}\n\n\t// NOTE: reader should set max memory bound with WithDecoderMaxMemory.\n\t// we can deduce it from maxChunkSize and protect against\n\t// malicious inputs.\n\tzstdReader, err = zstd.NewReader(\n\t\tnil,\n\t\tzstd.WithDecoderMaxMemory(32*maxChunkSize),\n\t)\n\tif err != nil {\n\t\t// configuring the reader wrong is a programmer error.\n\t\tpanic(err)\n\t}\n}\n\nvar (\n\t// AlgoMap is a map of available algorithms.\n\talgoMap = map[AlgorithmType]func() Algorithm{\n\t\tAlgoSnappy: func() Algorithm {\n\t\t\treturn snappyAlgo{}\n\t\t},\n\t\tAlgoLZ4: func() Algorithm {\n\t\t\t// TODO: we could configure compression level here.\n\t\t\treturn &lz4Algo{\n\t\t\t\tcompressor: &lz4.Compressor{},\n\t\t\t}\n\t\t},\n\t\tAlgoZstd: func() Algorithm {\n\t\t\treturn zstdAlgo{}\n\t\t},\n\t}\n\n\talgoToString = map[AlgorithmType]string{\n\t\tAlgoSnappy: \"snappy\",\n\t\tAlgoLZ4: \"lz4\",\n\t\tAlgoZstd: \"zstd\",\n\t}\n)\n\nfunc (a snappyAlgo) Encode(dst, src []byte) ([]byte, error) {\n\treturn snappy.Encode(dst, src), nil\n}\n\nfunc (a snappyAlgo) Decode(dst, src []byte) ([]byte, error) {\n\treturn snappy.Decode(dst, src)\n}\n\nfunc (a snappyAlgo) MaxEncodeBufferSize() int {\n\treturn snappy.MaxEncodedLen(maxChunkSize)\n}\n\n/////////////////////////\n\nfunc (a *lz4Algo) Encode(dst, src []byte) ([]byte, error) {\n\tn, err := a.compressor.CompressBlock(src, dst)\n\tif err != nil {\n\t\treturn dst[:n], err\n\t}\n\n\t// NOTE: n == 0 is returned when the data is not easy to compress\n\t// and the `dst` buf is too small to hold it. Since we always\n\t// supply a large enough buf this should not happen.\n\treturn dst[:n], nil\n}\n\nfunc (a *lz4Algo) Decode(dst, src []byte) ([]byte, error) {\n\tn, err := lz4.UncompressBlock(src, dst)\n\treturn dst[:n], err\n}\n\nfunc (a *lz4Algo) MaxEncodeBufferSize() int {\n\treturn lz4.CompressBlockBound(maxChunkSize)\n}\n\n/////////////////////////\n\nfunc (a zstdAlgo) Encode(dst, src []byte) ([]byte, error) {\n\treturn zstdWriter.EncodeAll(src, dst[:0]), nil\n}\n\nfunc (a zstdAlgo) Decode(dst, src []byte) ([]byte, error) {\n\treturn zstdReader.DecodeAll(src, dst[:0])\n}\n\nfunc (a zstdAlgo) MaxEncodeBufferSize() int {\n\t// TODO: Is there a better way to estimate?\n\treturn maxChunkSize * 2\n}\n\nfunc algorithmFromType(a AlgorithmType) (Algorithm, error) {\n\tnewAlgoFn, ok := algoMap[a]\n\tif !ok {\n\t\treturn nil, ErrBadAlgo\n\t}\n\n\treturn newAlgoFn(), nil\n}\n" }, { "path": "catfs/mio/compress/compress_test.go", "content": "package compress\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/sahib/brig/util\"\n\t\"github.com/sahib/brig/util/testutil\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nvar (\n\tTestOffsets = []int64{-1, -500, 0, 1, -C64K, -C32K, C64K - 1, C64K, C64K + 1, C32K - 1, C32K, C32K + 1, C64K - 5, C64K + 5, C32K - 5, C32K + 5}\n\tTestSizes = []int64{0, 1, 4096, C64K - 1, C64K, C64K + 1, C32K - 1, C32K, C32K + 1, C64K - 5, C64K + 5, C32K - 5, C32K + 5}\n\tCompressionAlgos = []AlgorithmType{AlgoLZ4, AlgoSnappy, AlgoZstd}\n)\n\nfunc openDest(t *testing.T, dest string) *os.File {\n\tif _, err := os.Stat(dest); !os.IsNotExist(err) && err != nil {\n\t\tt.Fatalf(\"Opening destination %v failed: %v\\n\", dest, err)\n\t}\n\tfd, err := os.OpenFile(dest, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0755)\n\tif err != nil {\n\t\tt.Fatalf(\"Opening source %v failed: %v\\n\", dest, err)\n\t}\n\treturn fd\n}\n\nfunc openSrc(t *testing.T, src string) *os.File {\n\tfd, err := os.Open(src)\n\tif err != nil {\n\t\tt.Fatalf(\"Opening source %v failed: %v\\n\", src, err)\n\t}\n\treturn fd\n}\n\nfunc createTempFile(t *testing.T) string {\n\tfd, err := ioutil.TempFile(\"\", \"brig-mio-compress\")\n\tpath := fd.Name()\n\trequire.Nil(t, err)\n\trequire.Nil(t, fd.Close())\n\treturn path\n}\n\nconst (\n\tC64K = 64 * 1024\n\tC32K = 32 * 1024\n)\n\nfunc TestCompressDecompress(t *testing.T) {\n\tt.Parallel()\n\n\tsizes := TestSizes\n\talgos := CompressionAlgos\n\n\tfor _, algo := range algos {\n\t\tfor _, size := range sizes {\n\t\t\tname := fmt.Sprintf(\"%s-size%d\", algo, size)\n\t\t\tt.Run(name, func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\n\t\t\t\ttestCompressDecompress(t, size, algo, true, true)\n\t\t\t\ttestCompressDecompress(t, size, algo, false, false)\n\t\t\t\ttestCompressDecompress(t, size, algo, true, false)\n\t\t\t\ttestCompressDecompress(t, size, algo, false, true)\n\t\t\t})\n\t\t}\n\t}\n}\n\nfunc testCompressDecompress(t *testing.T, size int64, algo AlgorithmType, useReadFrom, useWriteTo bool) {\n\t// Fake data file is written to disk,\n\t// as compression reader has to be a ReadSeeker.\n\tzipPath := createTempFile(t)\n\tdefer testutil.Remover(t, zipPath)\n\n\tdata := testutil.CreateDummyBuf(size)\n\tzipFileDest := openDest(t, zipPath)\n\n\t// Compress.\n\tw, err := NewWriter(zipFileDest, algo)\n\tif err != nil {\n\t\tt.Errorf(\"Writer init failed %v\", err)\n\t\treturn\n\n\t}\n\n\tif _, err := testutil.DumbCopy(w, bytes.NewReader(data), useReadFrom, useWriteTo); err != nil {\n\t\tt.Errorf(\"Compress failed %v\", err)\n\t\treturn\n\t}\n\n\tif err := w.Close(); err != nil {\n\t\tt.Errorf(\"Compression writer close failed: %v\", err)\n\t\treturn\n\t}\n\n\tif err := zipFileDest.Close(); err != nil {\n\t\tt.Errorf(\"close(zipFileDest) failed: %v\", err)\n\t\treturn\n\t}\n\n\t// Read compressed file into buffer.\n\tdataUncomp := bytes.NewBuffer(nil)\n\tdataFromZip := openSrc(t, zipPath)\n\n\t// Uncompress.\n\tr := NewReader(dataFromZip)\n\tif _, err := testutil.DumbCopy(dataUncomp, r, useReadFrom, useWriteTo); err != nil {\n\t\tt.Errorf(\"Decompression failed: %v\", err)\n\t\treturn\n\t}\n\tif err := dataFromZip.Close(); err != nil {\n\t\tt.Errorf(\"Zip close failed: %v\", err)\n\t\treturn\n\t}\n\n\t// Compare.\n\tgot, want := dataUncomp.Bytes(), data\n\tif !bytes.Equal(got, want) {\n\t\tt.Error(\"Uncompressed data and input data does not match.\")\n\t\tt.Errorf(\"\\tGOT: %v\", util.OmitBytes(got, 10))\n\t\tt.Errorf(\"\\tWANT: %v\", util.OmitBytes(want, 10))\n\t\treturn\n\t}\n}\n\nfunc TestSeek(t *testing.T) {\n\tt.Parallel()\n\n\tsizes := TestSizes\n\toffsets := TestOffsets\n\talgos := CompressionAlgos\n\tfor _, algo := range algos {\n\t\tfor _, size := range sizes {\n\t\t\tfor _, off := range offsets {\n\t\t\t\tname := fmt.Sprintf(\"%s-size%d-off%d\", algo, size, off)\n\t\t\t\tt.Run(name, func(t *testing.T) {\n\t\t\t\t\tt.Parallel()\n\n\t\t\t\t\ttestSeek(t, size, off, algo, false, false)\n\t\t\t\t\ttestSeek(t, size, off, algo, true, true)\n\t\t\t\t\ttestSeek(t, size, off, algo, false, true)\n\t\t\t\t\ttestSeek(t, size, off, algo, true, false)\n\t\t\t\t})\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc testSeek(t *testing.T, size, offset int64, algo AlgorithmType, useReadFrom, useWriteTo bool) {\n\t// Fake data file is written to disk,\n\t// as compression reader has to be a ReadSeeker.\n\tzipPath := createTempFile(t)\n\tdefer testutil.Remover(t, zipPath)\n\n\tdata := testutil.CreateDummyBuf(size)\n\tzipFileDest := openDest(t, zipPath)\n\n\t// Compress.\n\tw, err := NewWriter(zipFileDest, algo)\n\tif err != nil {\n\t\tt.Errorf(\"Writer init failed %v\", err)\n\t\treturn\n\t}\n\tif _, err := testutil.DumbCopy(w, bytes.NewReader(data), useReadFrom, useWriteTo); err != nil {\n\t\tt.Errorf(\"Compress failed %v\", err)\n\t\treturn\n\t}\n\tdefer testutil.Remover(t, zipPath)\n\n\tif err := w.Close(); err != nil {\n\t\tt.Errorf(\"Compression writer close failed: %v\", err)\n\t\treturn\n\t}\n\n\tif err := zipFileDest.Close(); err != nil {\n\t\tt.Errorf(\"close(zipFileDest) failed: %v\", err)\n\t\treturn\n\t}\n\n\t// Read compressed file into buffer.\n\tdataUncomp := bytes.NewBuffer(nil)\n\tdataFromZip := openSrc(t, zipPath)\n\tzr := NewReader(dataFromZip)\n\n\t// Set specific offset before read.\n\t_, err = zr.Seek(offset, io.SeekStart)\n\tif err == io.EOF && offset < size && offset > -1 {\n\t\tt.Errorf(\"Seek failed even with EOF: %d <= %d\", offset, size)\n\t\treturn\n\t}\n\tif err != io.EOF && err != nil {\n\t\tt.Errorf(\"Seek failed: %v\", err)\n\t\treturn\n\t}\n\n\t// Read starting at a specific offset.\n\tif _, err := testutil.DumbCopy(dataUncomp, zr, useReadFrom, useWriteTo); err != nil {\n\t\tt.Errorf(\"Decompression failed: %v\", err)\n\t\treturn\n\t}\n\tif err := dataFromZip.Close(); err != nil {\n\t\tt.Errorf(\"Zip close failed: %v\", err)\n\t\treturn\n\t}\n\n\t// Compare.\n\tmaxOffset := offset\n\tif offset > size {\n\t\tmaxOffset = size\n\t}\n\n\tif offset < 0 {\n\t\tmaxOffset = 0\n\t}\n\n\tgot, want := dataUncomp.Bytes(), data[maxOffset:]\n\tif !bytes.Equal(got, want) {\n\t\tt.Error(\"Uncompressed data and input data does not match.\")\n\t\tt.Errorf(\"\\tGOT: %v\", util.OmitBytes(got, 10))\n\t\tt.Errorf(\"\\tWANT: %v\", util.OmitBytes(want, 10))\n\t\treturn\n\t}\n}\n\nfunc TestReadItAllTwice(t *testing.T) {\n\tfor _, algo := range []AlgorithmType{AlgoLZ4, AlgoSnappy} {\n\t\tt.Run(fmt.Sprintf(\"%v\", algo), func(t *testing.T) {\n\t\t\tdata := testutil.CreateDummyBuf(2 * 4096)\n\t\t\tzipData, err := Pack(data, algo)\n\t\t\trequire.Nil(t, err)\n\n\t\t\tr := bytes.NewReader(zipData)\n\t\t\tzr := NewReader(r)\n\n\t\t\treadData1, err := ioutil.ReadAll(zr)\n\t\t\trequire.Nil(t, err)\n\n\t\t\tn, err := zr.Seek(0, io.SeekStart)\n\t\t\trequire.Nil(t, err)\n\t\t\trequire.Equal(t, n, int64(0))\n\n\t\t\treadData2, err := ioutil.ReadAll(zr)\n\t\t\trequire.Nil(t, err)\n\n\t\t\trequire.Equal(t, readData1, readData2)\n\t\t})\n\t}\n}\n\n// fuse will use Seek() to jump to each position.\n// when reading a complete file it will call seek before each read.\nfunc TestReadFuseLike(t *testing.T) {\n\talgo := AlgorithmType(AlgoSnappy)\n\n\tfor _, size := range TestSizes {\n\t\tt.Run(fmt.Sprintf(\"%v\", size), func(t *testing.T) {\n\t\t\tdata := testutil.CreateDummyBuf(size)\n\t\t\tcompressedData, err := Pack(data, algo)\n\t\t\trequire.Nil(t, err)\n\n\t\t\tr := NewReader(bytes.NewReader(compressedData))\n\t\t\tbufSize := 4096\n\t\t\tbuf := make([]byte, bufSize)\n\n\t\t\toffset := int64(0)\n\n\t\t\tfor {\n\t\t\t\tseekOffset, err := r.Seek(offset, io.SeekStart)\n\t\t\t\tif err != io.EOF {\n\t\t\t\t\trequire.Nil(t, err)\n\t\t\t\t\trequire.Equal(t, offset, seekOffset)\n\t\t\t\t}\n\n\t\t\t\tn, err := r.Read(buf)\n\t\t\t\tif err != io.EOF {\n\t\t\t\t\trequire.Nil(t, err)\n\t\t\t\t}\n\n\t\t\t\t// check that n returns something that makes sense:\n\t\t\t\trequire.Equal(t, util.Min(bufSize, len(data)-int(offset)), n)\n\t\t\t\trequire.Equal(t, data[offset:offset+int64(n)], buf[:n])\n\n\t\t\t\toffset += int64(n)\n\n\t\t\t\t// If this test goes into an endless loop: that's why.\n\t\t\t\tif err == io.EOF {\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestCheckSize(t *testing.T) {\n\tdata := testutil.CreateDummyBuf(6041)\n\tpackData, err := Pack(data, AlgoSnappy)\n\trequire.Nil(t, err)\n\n\tr := NewReader(bytes.NewReader(packData))\n\n\tsize, err := r.Seek(0, io.SeekEnd)\n\trequire.Nil(t, err)\n\trequire.Equal(t, int64(len(data)), size)\n\n\toff, err := r.Seek(0, io.SeekStart)\n\trequire.Nil(t, err)\n\trequire.Equal(t, int64(0), off)\n\n\tbuf := &bytes.Buffer{}\n\tn, err := io.Copy(buf, r)\n\trequire.Nil(t, err)\n\trequire.Equal(t, int64(len(data)), n)\n\trequire.Equal(t, data, buf.Bytes())\n}\n" }, { "path": "catfs/mio/compress/header.go", "content": "package compress\n\n// TODO: rename to header.go\nimport (\n\t\"bytes\"\n\t\"encoding/binary\"\n\t\"errors\"\n)\n\nvar (\n\t// ErrBadIndex is returned on invalid compression index.\n\tErrBadIndex = errors.New(\"broken compression index\")\n\n\t// ErrHeaderTooSmall is returned if the header is less than 10 bytes.\n\t// It usually indicates a broken file or a non-compressed file.\n\tErrHeaderTooSmall = errors.New(\"header is less than 10 bytes\")\n\n\t// ErrBadMagicNumber is returned if the first 8 bytes of the stream is not\n\t// the expected \"elchwald\".\n\tErrBadMagicNumber = errors.New(\"bad magic number in compressed stream\")\n\n\t// ErrBadAlgorithm is returned when the algorithm was either not present\n\t// or it had an invalid value\n\tErrBadAlgorithm = errors.New(\"invalid algorithm\")\n\n\t// ErrUnsupportedVersion is returned when we don't have a reader that\n\t// understands that format.\n\tErrUnsupportedVersion = errors.New(\"version of this format is not supported\")\n\n\t// MagicNumber is the magic number in front of a compressed stream\n\tMagicNumber = []byte(\"elchwald\")\n)\n\nconst (\n\tmaxChunkSize = 64 * 1024\n\tindexChunkSize = 16\n\ttrailerSize = 12\n\theaderSize = 12\n\tcurrentVersion = 1\n)\n\n// record structure reprenents a offset mapping {uncompressed offset, compressedOffset}.\n// A chunk of maxChunkSize is defined by two records. The size of a specific\n// record can be determinated by a simple substitution of two record offsets.\ntype record struct {\n\trawOff int64\n\tzipOff int64\n}\n\n// trailer holds basic information about the compressed file.\ntype trailer struct {\n\tchunksize uint32\n\tindexSize uint64\n}\n\nfunc (t *trailer) marshal(buf []byte) {\n\tbinary.LittleEndian.PutUint32(buf[0:4], t.chunksize)\n\tbinary.LittleEndian.PutUint64(buf[4:12], t.indexSize)\n}\n\nfunc (t *trailer) unmarshal(buf []byte) {\n\tt.chunksize = binary.LittleEndian.Uint32(buf[0:4])\n\tt.indexSize = binary.LittleEndian.Uint64(buf[4:12])\n}\n\nfunc (rc *record) marshal(buf []byte) {\n\tbinary.LittleEndian.PutUint64(buf[0:8], uint64(rc.rawOff))\n\tbinary.LittleEndian.PutUint64(buf[8:16], uint64(rc.zipOff))\n}\n\nfunc (rc *record) unmarshal(buf []byte) {\n\trc.rawOff = int64(binary.LittleEndian.Uint64(buf[0:8]))\n\trc.zipOff = int64(binary.LittleEndian.Uint64(buf[8:16]))\n}\n\ntype header struct {\n\talgo AlgorithmType\n\tversion uint16\n}\n\nfunc makeHeader(algo AlgorithmType, version byte) []byte {\n\talgoField := make([]byte, 2)\n\tbinary.LittleEndian.PutUint16(algoField, uint16(algo))\n\n\tversionField := make([]byte, 2)\n\tbinary.LittleEndian.PutUint16(versionField, uint16(version))\n\n\tsuffix := append(versionField, algoField...)\n\treturn append(MagicNumber, suffix...)\n}\n\nfunc readHeader(bheader []byte) (*header, error) {\n\tif len(bheader) < 10 {\n\t\treturn nil, ErrHeaderTooSmall\n\t}\n\n\tif !bytes.Equal(bheader[:len(MagicNumber)], MagicNumber) {\n\t\treturn nil, ErrBadMagicNumber\n\t}\n\n\t// This version only understands itself currently:\n\tversion := binary.LittleEndian.Uint16(bheader[8:10])\n\tif version != currentVersion {\n\t\treturn nil, ErrUnsupportedVersion\n\t}\n\n\tif len(bheader) < 12 {\n\t\treturn nil, ErrBadAlgorithm\n\t}\n\n\talgo := AlgorithmType(binary.LittleEndian.Uint16(bheader[10:12]))\n\tif !algo.IsValid() {\n\t\treturn nil, ErrBadAlgorithm\n\t}\n\n\treturn &header{\n\t\talgo: algo,\n\t\tversion: version,\n\t}, nil\n}\n\n// Pack compresses `data` with `algo` and returns the resulting data.\n// This is a convenience method meant to be used for small data packages.\nfunc Pack(data []byte, algo AlgorithmType) ([]byte, error) {\n\tzipBuf := &bytes.Buffer{}\n\tzipW, err := NewWriter(zipBuf, algo)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif _, err := zipW.ReadFrom(bytes.NewReader(data)); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := zipW.Close(); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn zipBuf.Bytes(), nil\n}\n\n// Unpack unpacks `data` and returns the decompressed data.\n// The algorithm is read from the data itself.\n// This is a convinience method meant to be used for small data packages.\nfunc Unpack(data []byte) ([]byte, error) {\n\tbuf := &bytes.Buffer{}\n\tif _, err := NewReader(bytes.NewReader(data)).WriteTo(buf); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn buf.Bytes(), nil\n}\n" }, { "path": "catfs/mio/compress/heuristic.go", "content": "package compress\n\nimport (\n\t\"mime\"\n\t\"net/http\"\n\t\"path/filepath\"\n\t\"strings\"\n\n\t\"github.com/sdemontfort/go-mimemagic\"\n)\n\nvar (\n\t// TextFileExtensions not covered by mime.TypeByExtension\n\tTextFileExtensions = map[string]bool{\n\t\t\".go\": true,\n\t\t\".json\": true,\n\t\t\".yaml\": true,\n\t\t\".xml\": true,\n\t\t\".txt\": true,\n\t}\n)\n\nconst (\n\t// HeaderSizeThreshold is the number of bytes needed to enable compression at all.\n\tHeaderSizeThreshold = 2048\n)\n\nfunc guessMime(path string, buf []byte) string {\n\thttpMatch := http.DetectContentType(buf)\n\tif httpMatch != \"application/octet-stream\" {\n\t\treturn httpMatch\n\t}\n\n\t// try to guess it from the buffer we pass:\n\tmatch := mimemagic.Match(\"\", buf)\n\tif match == \"\" {\n\t\t// try to guess it from the file path:\n\t\tmatch = mime.TypeByExtension(filepath.Ext(path))\n\t}\n\n\t// handle a few edge cases:\n\tif TextFileExtensions[filepath.Ext(path)] {\n\t\treturn \"text/plain\"\n\t}\n\n\treturn match\n}\n\nfunc isCompressible(mimetype string) bool {\n\tif strings.HasPrefix(mimetype, \"text/\") {\n\t\treturn true\n\t}\n\n\treturn CompressibleMapping[mimetype]\n}\n\n// GuessAlgorithm takes the path name and the header data of it\n// and tries to guess a suitable compression algorithm.\nfunc GuessAlgorithm(path string, header []byte) (AlgorithmType, error) {\n\tif len(header) < HeaderSizeThreshold {\n\t\treturn AlgoUnknown, nil\n\t}\n\n\tmime := guessMime(path, header)\n\tif mime == \"\" {\n\t\t// the guesses below work only when mime is known\n\t\treturn AlgoSnappy, nil\n\t}\n\n\tcompressible := isCompressible(mime)\n\tif !compressible {\n\t\treturn AlgoUnknown, nil\n\t}\n\n\t// text like files probably deserve some thorough compression:\n\tif strings.HasPrefix(mime, \"text/\") {\n\t\treturn AlgoLZ4, nil\n\t}\n\n\t// fallback to snappy for generic files:\n\treturn AlgoSnappy, nil\n}\n" }, { "path": "catfs/mio/compress/heuristic_test.go", "content": "package compress\n\nimport (\n\t\"testing\"\n\n\t\"github.com/sahib/brig/util/testutil\"\n)\n\ntype testCase struct {\n\tpath string\n\theader []byte\n\texpectedAlgo AlgorithmType\n}\n\nvar (\n\ttestCases = []testCase{\n\t\t{\n\t\t\t\"1.txt\",\n\t\t\ttestutil.CreateDummyBuf(HeaderSizeThreshold - 1),\n\t\t\tAlgoUnknown,\n\t\t}, {\n\t\t\t\"2.txt\",\n\t\t\ttestutil.CreateDummyBuf(HeaderSizeThreshold),\n\t\t\tAlgoLZ4,\n\t\t}, {\n\t\t\t\"3.opus\",\n\t\t\tappend(\n\t\t\t\t[]byte{0x4f, 0x67, 0x67, 0x53},\n\t\t\t\ttestutil.CreateDummyBuf(HeaderSizeThreshold)...,\n\t\t\t),\n\t\t\tAlgoUnknown,\n\t\t}, {\n\t\t\t\"4.zip\",\n\t\t\tappend(\n\t\t\t\t[]byte{0x50, 0x4b, 0x3, 0x4},\n\t\t\t\ttestutil.CreateDummyBuf(HeaderSizeThreshold)...,\n\t\t\t),\n\t\t\tAlgoUnknown,\n\t\t},\n\t}\n)\n\nfunc TestChooseCompressAlgo(t *testing.T) {\n\tt.Parallel()\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.path, func(t *testing.T) {\n\t\t\tif algo, err := GuessAlgorithm(tc.path, tc.header); err != nil {\n\t\t\t\tt.Errorf(\"Error: %v\", err)\n\t\t\t} else if algo != tc.expectedAlgo {\n\t\t\t\tt.Errorf(\n\t\t\t\t\t\"For path '%s' expected '%s', got '%s'\",\n\t\t\t\t\ttc.path,\n\t\t\t\t\talgoToString[tc.expectedAlgo],\n\t\t\t\t\talgoToString[algo],\n\t\t\t\t)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "catfs/mio/compress/mime_db.go", "content": "package compress\n\n// CompressibleMapping maps between mime types and a bool indicating\n// if they're compressible. Choice of Algorithm comes later.\n//\n// This was converted from this mime db:\n// https://cdn.rawgit.com/jshttp/mime-db/master/db.json\nvar CompressibleMapping = map[string]bool{\n\t\"application/3gpdash-qoe-report+xml\": true,\n\t\"application/3gpp-ims+xml\": true,\n\t\"application/activity+json\": true,\n\t\"application/alto-costmap+json\": true,\n\t\"application/alto-costmapfilter+json\": true,\n\t\"application/alto-directory+json\": true,\n\t\"application/alto-endpointcost+json\": true,\n\t\"application/alto-endpointcostparams+json\": true,\n\t\"application/alto-endpointprop+json\": true,\n\t\"application/alto-endpointpropparams+json\": true,\n\t\"application/alto-error+json\": true,\n\t\"application/alto-networkmap+json\": true,\n\t\"application/alto-networkmapfilter+json\": true,\n\t\"application/atom+xml\": true,\n\t\"application/atomcat+xml\": true,\n\t\"application/atomdeleted+xml\": true,\n\t\"application/atomsvc+xml\": true,\n\t\"application/auth-policy+xml\": true,\n\t\"application/bdoc\": false,\n\t\"application/beep+xml\": true,\n\t\"application/calendar+json\": true,\n\t\"application/calendar+xml\": true,\n\t\"application/ccmp+xml\": true,\n\t\"application/ccxml+xml\": true,\n\t\"application/cdfx+xml\": true,\n\t\"application/cea-2018+xml\": true,\n\t\"application/cellml+xml\": true,\n\t\"application/clue_info+xml\": true,\n\t\"application/cnrp+xml\": true,\n\t\"application/coap-group+json\": true,\n\t\"application/conference-info+xml\": true,\n\t\"application/cpl+xml\": true,\n\t\"application/csta+xml\": true,\n\t\"application/cstadata+xml\": true,\n\t\"application/csvm+json\": true,\n\t\"application/dart\": true,\n\t\"application/dash+xml\": true,\n\t\"application/davmount+xml\": true,\n\t\"application/dialog-info+xml\": true,\n\t\"application/dicom+json\": true,\n\t\"application/dicom+xml\": true,\n\t\"application/docbook+xml\": true,\n\t\"application/dskpp+xml\": true,\n\t\"application/dssc+xml\": true,\n\t\"application/ecmascript\": true,\n\t\"application/edi-x12\": false,\n\t\"application/edifact\": false,\n\t\"application/emergencycalldata.comment+xml\": true,\n\t\"application/emergencycalldata.control+xml\": true,\n\t\"application/emergencycalldata.deviceinfo+xml\": true,\n\t\"application/emergencycalldata.providerinfo+xml\": true,\n\t\"application/emergencycalldata.serviceinfo+xml\": true,\n\t\"application/emergencycalldata.subscriberinfo+xml\": true,\n\t\"application/emergencycalldata.veds+xml\": true,\n\t\"application/emma+xml\": true,\n\t\"application/emotionml+xml\": true,\n\t\"application/epp+xml\": true,\n\t\"application/fdt+xml\": true,\n\t\"application/fhir+json\": true,\n\t\"application/fhir+xml\": true,\n\t\"application/fido.trusted-apps+json\": true,\n\t\"application/font-woff\": false,\n\t\"application/framework-attributes+xml\": true,\n\t\"application/geo+json\": true,\n\t\"application/geoxacml+xml\": true,\n\t\"application/gml+xml\": true,\n\t\"application/gpx+xml\": true,\n\t\"application/gzip\": false,\n\t\"application/held+xml\": true,\n\t\"application/ibe-key-request+xml\": true,\n\t\"application/ibe-pkg-reply+xml\": true,\n\t\"application/im-iscomposing+xml\": true,\n\t\"application/inkml+xml\": true,\n\t\"application/its+xml\": true,\n\t\"application/java-archive\": false,\n\t\"application/java-serialized-object\": false,\n\t\"application/java-vm\": false,\n\t\"application/javascript\": true,\n\t\"application/jf2feed+json\": true,\n\t\"application/jose+json\": true,\n\t\"application/jrd+json\": true,\n\t\"application/json\": true,\n\t\"application/json-patch+json\": true,\n\t\"application/jsonml+json\": true,\n\t\"application/jwk+json\": true,\n\t\"application/jwk-set+json\": true,\n\t\"application/kpml-request+xml\": true,\n\t\"application/kpml-response+xml\": true,\n\t\"application/ld+json\": true,\n\t\"application/lgr+xml\": true,\n\t\"application/load-control+xml\": true,\n\t\"application/lost+xml\": true,\n\t\"application/lostsync+xml\": true,\n\t\"application/mads+xml\": true,\n\t\"application/manifest+json\": true,\n\t\"application/marcxml+xml\": true,\n\t\"application/mathml+xml\": true,\n\t\"application/mathml-content+xml\": true,\n\t\"application/mathml-presentation+xml\": true,\n\t\"application/mbms-associated-procedure-description+xml\": true,\n\t\"application/mbms-deregister+xml\": true,\n\t\"application/mbms-envelope+xml\": true,\n\t\"application/mbms-msk+xml\": true,\n\t\"application/mbms-msk-response+xml\": true,\n\t\"application/mbms-protection-description+xml\": true,\n\t\"application/mbms-reception-report+xml\": true,\n\t\"application/mbms-register+xml\": true,\n\t\"application/mbms-register-response+xml\": true,\n\t\"application/mbms-schedule+xml\": true,\n\t\"application/mbms-user-service-description+xml\": true,\n\t\"application/media-policy-dataset+xml\": true,\n\t\"application/media_control+xml\": true,\n\t\"application/mediaservercontrol+xml\": true,\n\t\"application/merge-patch+json\": true,\n\t\"application/metalink+xml\": true,\n\t\"application/metalink4+xml\": true,\n\t\"application/mets+xml\": true,\n\t\"application/mmt-usd+xml\": true,\n\t\"application/mods+xml\": true,\n\t\"application/mrb-consumer+xml\": true,\n\t\"application/mrb-publish+xml\": true,\n\t\"application/msc-ivr+xml\": true,\n\t\"application/msc-mixer+xml\": true,\n\t\"application/msword\": false,\n\t\"application/mud+json\": true,\n\t\"application/nlsml+xml\": true,\n\t\"application/octet-stream\": false,\n\t\"application/oebps-package+xml\": true,\n\t\"application/ogg\": false,\n\t\"application/omdoc+xml\": true,\n\t\"application/p2p-overlay+xml\": true,\n\t\"application/patch-ops-error+xml\": true,\n\t\"application/pdf\": false,\n\t\"application/pgp-encrypted\": false,\n\t\"application/pidf+xml\": true,\n\t\"application/pidf-diff+xml\": true,\n\t\"application/pls+xml\": true,\n\t\"application/poc-settings+xml\": true,\n\t\"application/postscript\": true,\n\t\"application/ppsp-tracker+json\": true,\n\t\"application/problem+json\": true,\n\t\"application/problem+xml\": true,\n\t\"application/provenance+xml\": true,\n\t\"application/prs.xsf+xml\": true,\n\t\"application/pskc+xml\": true,\n\t\"application/raml+yaml\": true,\n\t\"application/rdap+json\": true,\n\t\"application/rdf+xml\": true,\n\t\"application/reginfo+xml\": true,\n\t\"application/reputon+json\": true,\n\t\"application/resource-lists+xml\": true,\n\t\"application/resource-lists-diff+xml\": true,\n\t\"application/rfc+xml\": true,\n\t\"application/rlmi+xml\": true,\n\t\"application/rls-services+xml\": true,\n\t\"application/route-apd+xml\": true,\n\t\"application/route-s-tsid+xml\": true,\n\t\"application/route-usd+xml\": true,\n\t\"application/rsd+xml\": true,\n\t\"application/rss+xml\": true,\n\t\"application/rtf\": true,\n\t\"application/samlassertion+xml\": true,\n\t\"application/samlmetadata+xml\": true,\n\t\"application/sbml+xml\": true,\n\t\"application/scaip+xml\": true,\n\t\"application/scim+json\": true,\n\t\"application/sep+xml\": true,\n\t\"application/shf+xml\": true,\n\t\"application/simple-filter+xml\": true,\n\t\"application/smil+xml\": true,\n\t\"application/soap+xml\": true,\n\t\"application/sparql-results+xml\": true,\n\t\"application/spirits-event+xml\": true,\n\t\"application/srgs+xml\": true,\n\t\"application/sru+xml\": true,\n\t\"application/ssdl+xml\": true,\n\t\"application/ssml+xml\": true,\n\t\"application/tar\": true,\n\t\"application/tei+xml\": true,\n\t\"application/thraud+xml\": true,\n\t\"application/ttml+xml\": true,\n\t\"application/urc-grpsheet+xml\": true,\n\t\"application/urc-ressheet+xml\": true,\n\t\"application/urc-targetdesc+xml\": true,\n\t\"application/urc-uisocketdesc+xml\": true,\n\t\"application/vcard+json\": true,\n\t\"application/vcard+xml\": true,\n\t\"application/vnd.1000minds.decision-model+xml\": true,\n\t\"application/vnd.3gpp-prose+xml\": true,\n\t\"application/vnd.3gpp-prose-pc3ch+xml\": true,\n\t\"application/vnd.3gpp.access-transfer-events+xml\": true,\n\t\"application/vnd.3gpp.bsf+xml\": true,\n\t\"application/vnd.3gpp.gmop+xml\": true,\n\t\"application/vnd.3gpp.mcptt-affiliation-command+xml\": true,\n\t\"application/vnd.3gpp.mcptt-floor-request+xml\": true,\n\t\"application/vnd.3gpp.mcptt-info+xml\": true,\n\t\"application/vnd.3gpp.mcptt-location-info+xml\": true,\n\t\"application/vnd.3gpp.mcptt-mbms-usage-info+xml\": true,\n\t\"application/vnd.3gpp.mcptt-signed+xml\": true,\n\t\"application/vnd.3gpp.mid-call+xml\": true,\n\t\"application/vnd.3gpp.sms+xml\": true,\n\t\"application/vnd.3gpp.srvcc-ext+xml\": true,\n\t\"application/vnd.3gpp.srvcc-info+xml\": true,\n\t\"application/vnd.3gpp.state-and-event-info+xml\": true,\n\t\"application/vnd.3gpp.ussd+xml\": true,\n\t\"application/vnd.3gpp2.bcmcsinfo+xml\": true,\n\t\"application/vnd.adobe.xdp+xml\": true,\n\t\"application/vnd.amadeus+json\": true,\n\t\"application/vnd.amundsen.maze+xml\": true,\n\t\"application/vnd.android.package-archive\": false,\n\t\"application/vnd.api+json\": true,\n\t\"application/vnd.apothekende.reservation+json\": true,\n\t\"application/vnd.apple.installer+xml\": true,\n\t\"application/vnd.apple.pkpass\": false,\n\t\"application/vnd.avalon+json\": true,\n\t\"application/vnd.avistar+xml\": true,\n\t\"application/vnd.balsamiq.bmml+xml\": true,\n\t\"application/vnd.bbf.usp.msg+json\": true,\n\t\"application/vnd.bekitzur-stech+json\": true,\n\t\"application/vnd.biopax.rdf+xml\": true,\n\t\"application/vnd.capasystems-pg+json\": true,\n\t\"application/vnd.chemdraw+xml\": true,\n\t\"application/vnd.citationstyles.style+xml\": true,\n\t\"application/vnd.collection+json\": true,\n\t\"application/vnd.collection.doc+json\": true,\n\t\"application/vnd.collection.next+json\": true,\n\t\"application/vnd.coreos.ignition+json\": true,\n\t\"application/vnd.criticaltools.wbs+xml\": true,\n\t\"application/vnd.ctct.ws+xml\": true,\n\t\"application/vnd.cyan.dean.root+xml\": true,\n\t\"application/vnd.dart\": true,\n\t\"application/vnd.datapackage+json\": true,\n\t\"application/vnd.dataresource+json\": true,\n\t\"application/vnd.dece.ttml+xml\": true,\n\t\"application/vnd.dm.delegation+xml\": true,\n\t\"application/vnd.document+json\": true,\n\t\"application/vnd.drive+json\": true,\n\t\"application/vnd.dvb.notif-aggregate-root+xml\": true,\n\t\"application/vnd.dvb.notif-container+xml\": true,\n\t\"application/vnd.dvb.notif-generic+xml\": true,\n\t\"application/vnd.dvb.notif-ia-msglist+xml\": true,\n\t\"application/vnd.dvb.notif-ia-registration-request+xml\": true,\n\t\"application/vnd.dvb.notif-ia-registration-response+xml\": true,\n\t\"application/vnd.dvb.notif-init+xml\": true,\n\t\"application/vnd.emclient.accessrequest+xml\": true,\n\t\"application/vnd.eprints.data+xml\": true,\n\t\"application/vnd.eszigno3+xml\": true,\n\t\"application/vnd.etsi.aoc+xml\": true,\n\t\"application/vnd.etsi.cug+xml\": true,\n\t\"application/vnd.etsi.iptvcommand+xml\": true,\n\t\"application/vnd.etsi.iptvdiscovery+xml\": true,\n\t\"application/vnd.etsi.iptvprofile+xml\": true,\n\t\"application/vnd.etsi.iptvsad-bc+xml\": true,\n\t\"application/vnd.etsi.iptvsad-cod+xml\": true,\n\t\"application/vnd.etsi.iptvsad-npvr+xml\": true,\n\t\"application/vnd.etsi.iptvservice+xml\": true,\n\t\"application/vnd.etsi.iptvsync+xml\": true,\n\t\"application/vnd.etsi.iptvueprofile+xml\": true,\n\t\"application/vnd.etsi.mcid+xml\": true,\n\t\"application/vnd.etsi.overload-control-policy-dataset+xml\": true,\n\t\"application/vnd.etsi.pstn+xml\": true,\n\t\"application/vnd.etsi.sci+xml\": true,\n\t\"application/vnd.etsi.simservs+xml\": true,\n\t\"application/vnd.etsi.tsl+xml\": true,\n\t\"application/vnd.geo+json\": true,\n\t\"application/vnd.geocube+xml\": true,\n\t\"application/vnd.google-apps.document\": false,\n\t\"application/vnd.google-apps.presentation\": false,\n\t\"application/vnd.google-apps.spreadsheet\": false,\n\t\"application/vnd.google-earth.kml+xml\": true,\n\t\"application/vnd.google-earth.kmz\": false,\n\t\"application/vnd.gov.sk.e-form+xml\": true,\n\t\"application/vnd.gov.sk.xmldatacontainer+xml\": true,\n\t\"application/vnd.hal+json\": true,\n\t\"application/vnd.hal+xml\": true,\n\t\"application/vnd.handheld-entertainment+xml\": true,\n\t\"application/vnd.hc+json\": true,\n\t\"application/vnd.heroku+json\": true,\n\t\"application/vnd.hyper+json\": true,\n\t\"application/vnd.hyper-item+json\": true,\n\t\"application/vnd.hyperdrive+json\": true,\n\t\"application/vnd.ims.lis.v2.result+json\": true,\n\t\"application/vnd.ims.lti.v2.toolconsumerprofile+json\": true,\n\t\"application/vnd.ims.lti.v2.toolproxy+json\": true,\n\t\"application/vnd.ims.lti.v2.toolproxy.id+json\": true,\n\t\"application/vnd.ims.lti.v2.toolsettings+json\": true,\n\t\"application/vnd.ims.lti.v2.toolsettings.simple+json\": true,\n\t\"application/vnd.informedcontrol.rms+xml\": true,\n\t\"application/vnd.infotech.project+xml\": true,\n\t\"application/vnd.iptc.g2.catalogitem+xml\": true,\n\t\"application/vnd.iptc.g2.conceptitem+xml\": true,\n\t\"application/vnd.iptc.g2.knowledgeitem+xml\": true,\n\t\"application/vnd.iptc.g2.newsitem+xml\": true,\n\t\"application/vnd.iptc.g2.newsmessage+xml\": true,\n\t\"application/vnd.iptc.g2.packageitem+xml\": true,\n\t\"application/vnd.iptc.g2.planningitem+xml\": true,\n\t\"application/vnd.irepository.package+xml\": true,\n\t\"application/vnd.las.las+json\": true,\n\t\"application/vnd.las.las+xml\": true,\n\t\"application/vnd.liberty-request+xml\": true,\n\t\"application/vnd.llamagraphics.life-balance.exchange+xml\": true,\n\t\"application/vnd.marlin.drm.actiontoken+xml\": true,\n\t\"application/vnd.marlin.drm.conftoken+xml\": true,\n\t\"application/vnd.marlin.drm.license+xml\": true,\n\t\"application/vnd.mason+json\": true,\n\t\"application/vnd.micro+json\": true,\n\t\"application/vnd.miele+json\": true,\n\t\"application/vnd.mozilla.xul+xml\": true,\n\t\"application/vnd.ms-excel\": false,\n\t\"application/vnd.ms-fontobject\": true,\n\t\"application/vnd.ms-office.activex+xml\": true,\n\t\"application/vnd.ms-opentype\": true,\n\t\"application/vnd.ms-outlook\": false,\n\t\"application/vnd.ms-playready.initiator+xml\": true,\n\t\"application/vnd.ms-powerpoint\": false,\n\t\"application/vnd.ms-printdevicecapabilities+xml\": true,\n\t\"application/vnd.ms-printing.printticket+xml\": true,\n\t\"application/vnd.ms-printschematicket+xml\": true,\n\t\"application/vnd.ms-xpsdocument\": false,\n\t\"application/vnd.nearst.inv+json\": true,\n\t\"application/vnd.nokia.conml+xml\": true,\n\t\"application/vnd.nokia.iptv.config+xml\": true,\n\t\"application/vnd.nokia.landmark+xml\": true,\n\t\"application/vnd.nokia.landmarkcollection+xml\": true,\n\t\"application/vnd.nokia.n-gage.ac+xml\": true,\n\t\"application/vnd.nokia.pcd+xml\": true,\n\t\"application/vnd.oasis.opendocument.graphics\": false,\n\t\"application/vnd.oasis.opendocument.presentation\": false,\n\t\"application/vnd.oasis.opendocument.spreadsheet\": false,\n\t\"application/vnd.oasis.opendocument.text\": false,\n\t\"application/vnd.oftn.l10n+json\": true,\n\t\"application/vnd.oipf.contentaccessdownload+xml\": true,\n\t\"application/vnd.oipf.contentaccessstreaming+xml\": true,\n\t\"application/vnd.oipf.dae.svg+xml\": true,\n\t\"application/vnd.oipf.dae.xhtml+xml\": true,\n\t\"application/vnd.oipf.mippvcontrolmessage+xml\": true,\n\t\"application/vnd.oipf.spdiscovery+xml\": true,\n\t\"application/vnd.oipf.spdlist+xml\": true,\n\t\"application/vnd.oipf.ueprofile+xml\": true,\n\t\"application/vnd.oipf.userprofile+xml\": true,\n\t\"application/vnd.oma.bcast.associated-procedure-parameter+xml\": true,\n\t\"application/vnd.oma.bcast.drm-trigger+xml\": true,\n\t\"application/vnd.oma.bcast.imd+xml\": true,\n\t\"application/vnd.oma.bcast.notification+xml\": true,\n\t\"application/vnd.oma.bcast.sgdd+xml\": true,\n\t\"application/vnd.oma.bcast.smartcard-trigger+xml\": true,\n\t\"application/vnd.oma.bcast.sprov+xml\": true,\n\t\"application/vnd.oma.cab-address-book+xml\": true,\n\t\"application/vnd.oma.cab-feature-handler+xml\": true,\n\t\"application/vnd.oma.cab-pcc+xml\": true,\n\t\"application/vnd.oma.cab-subs-invite+xml\": true,\n\t\"application/vnd.oma.cab-user-prefs+xml\": true,\n\t\"application/vnd.oma.dd2+xml\": true,\n\t\"application/vnd.oma.drm.risd+xml\": true,\n\t\"application/vnd.oma.group-usage-list+xml\": true,\n\t\"application/vnd.oma.lwm2m+json\": true,\n\t\"application/vnd.oma.pal+xml\": true,\n\t\"application/vnd.oma.poc.detailed-progress-report+xml\": true,\n\t\"application/vnd.oma.poc.final-report+xml\": true,\n\t\"application/vnd.oma.poc.groups+xml\": true,\n\t\"application/vnd.oma.poc.invocation-descriptor+xml\": true,\n\t\"application/vnd.oma.poc.optimized-progress-report+xml\": true,\n\t\"application/vnd.oma.scidm.messages+xml\": true,\n\t\"application/vnd.oma.xcap-directory+xml\": true,\n\t\"application/vnd.omads-email+xml\": true,\n\t\"application/vnd.omads-file+xml\": true,\n\t\"application/vnd.omads-folder+xml\": true,\n\t\"application/vnd.openblox.game+xml\": true,\n\t\"application/vnd.openstreetmap.data+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.custom-properties+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.customxmlproperties+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.drawing+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.drawingml.chart+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.drawingml.chartshapes+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.drawingml.diagramcolors+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.drawingml.diagramdata+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.drawingml.diagramlayout+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.drawingml.diagramstyle+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.extended-properties+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.presentationml.commentauthors+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.presentationml.comments+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.presentationml.handoutmaster+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.presentationml.notesmaster+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.presentationml.notesslide+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.presentationml.presentation\": false,\n\t\"application/vnd.openxmlformats-officedocument.presentationml.presentation.main+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.presentationml.presprops+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.presentationml.slide+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.presentationml.slidelayout+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.presentationml.slidemaster+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.presentationml.slideshow.main+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.presentationml.slideupdateinfo+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.presentationml.tablestyles+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.presentationml.tags+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.presentationml.template.main+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.presentationml.viewprops+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.calcchain+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.chartsheet+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.comments+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.connections+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.dialogsheet+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.externallink+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.pivotcachedefinition+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.pivotcacherecords+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.pivottable+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.querytable+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.revisionheaders+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.revisionlog+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.sharedstrings+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet\": false,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet.main+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.sheetmetadata+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.styles+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.table+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.tablesinglecells+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.template.main+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.usernames+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.volatiledependencies+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.spreadsheetml.worksheet+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.theme+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.themeoverride+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.wordprocessingml.comments+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.wordprocessingml.document\": false,\n\t\"application/vnd.openxmlformats-officedocument.wordprocessingml.document.glossary+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.wordprocessingml.endnotes+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.wordprocessingml.fonttable+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.wordprocessingml.footer+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.wordprocessingml.footnotes+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.wordprocessingml.numbering+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.wordprocessingml.settings+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.wordprocessingml.styles+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.wordprocessingml.template.main+xml\": true,\n\t\"application/vnd.openxmlformats-officedocument.wordprocessingml.websettings+xml\": true,\n\t\"application/vnd.openxmlformats-package.core-properties+xml\": true,\n\t\"application/vnd.openxmlformats-package.digital-signature-xmlsignature+xml\": true,\n\t\"application/vnd.openxmlformats-package.relationships+xml\": true,\n\t\"application/vnd.oracle.resource+json\": true,\n\t\"application/vnd.otps.ct-kip+xml\": true,\n\t\"application/vnd.pagerduty+json\": true,\n\t\"application/vnd.paos+xml\": true,\n\t\"application/vnd.poc.group-advertisement+xml\": true,\n\t\"application/vnd.pwg-xhtml-print+xml\": true,\n\t\"application/vnd.radisys.moml+xml\": true,\n\t\"application/vnd.radisys.msml+xml\": true,\n\t\"application/vnd.radisys.msml-audit+xml\": true,\n\t\"application/vnd.radisys.msml-audit-conf+xml\": true,\n\t\"application/vnd.radisys.msml-audit-conn+xml\": true,\n\t\"application/vnd.radisys.msml-audit-dialog+xml\": true,\n\t\"application/vnd.radisys.msml-audit-stream+xml\": true,\n\t\"application/vnd.radisys.msml-conf+xml\": true,\n\t\"application/vnd.radisys.msml-dialog+xml\": true,\n\t\"application/vnd.radisys.msml-dialog-base+xml\": true,\n\t\"application/vnd.radisys.msml-dialog-fax-detect+xml\": true,\n\t\"application/vnd.radisys.msml-dialog-fax-sendrecv+xml\": true,\n\t\"application/vnd.radisys.msml-dialog-group+xml\": true,\n\t\"application/vnd.radisys.msml-dialog-speech+xml\": true,\n\t\"application/vnd.radisys.msml-dialog-transform+xml\": true,\n\t\"application/vnd.recordare.musicxml+xml\": true,\n\t\"application/vnd.restful+json\": true,\n\t\"application/vnd.route66.link66+xml\": true,\n\t\"application/vnd.shootproof+json\": true,\n\t\"application/vnd.siren+json\": true,\n\t\"application/vnd.software602.filler.form+xml\": true,\n\t\"application/vnd.solent.sdkm+xml\": true,\n\t\"application/vnd.sun.wadl+xml\": true,\n\t\"application/vnd.syncml+xml\": true,\n\t\"application/vnd.syncml.dm+xml\": true,\n\t\"application/vnd.syncml.dmddf+xml\": true,\n\t\"application/vnd.syncml.dmtnds+xml\": true,\n\t\"application/vnd.tableschema+json\": true,\n\t\"application/vnd.think-cell.ppttc+json\": true,\n\t\"application/vnd.tmd.mediaflex.api+xml\": true,\n\t\"application/vnd.uoml+xml\": true,\n\t\"application/vnd.vel+json\": true,\n\t\"application/vnd.wv.csp+xml\": true,\n\t\"application/vnd.wv.ssp+xml\": true,\n\t\"application/vnd.xacml+json\": true,\n\t\"application/vnd.xmi+xml\": true,\n\t\"application/vnd.yamaha.openscoreformat.osfpvg+xml\": true,\n\t\"application/vnd.zzazz.deck+xml\": true,\n\t\"application/voicexml+xml\": true,\n\t\"application/voucher-cms+json\": true,\n\t\"application/wasm\": true,\n\t\"application/watcherinfo+xml\": true,\n\t\"application/webpush-options+json\": true,\n\t\"application/wsdl+xml\": true,\n\t\"application/wspolicy+xml\": true,\n\t\"application/x-7z-compressed\": false,\n\t\"application/x-arj\": false,\n\t\"application/x-bdoc\": false,\n\t\"application/x-bzip\": false,\n\t\"application/x-bzip2\": false,\n\t\"application/x-deb\": false,\n\t\"application/x-dtbncx+xml\": true,\n\t\"application/x-dtbook+xml\": true,\n\t\"application/x-dtbresource+xml\": true,\n\t\"application/x-dvi\": false,\n\t\"application/x-httpd-php\": true,\n\t\"application/x-java-jnlp-file\": false,\n\t\"application/x-javascript\": true,\n\t\"application/x-latex\": false,\n\t\"application/x-mpegurl\": false,\n\t\"application/x-ns-proxy-autoconfig\": true,\n\t\"application/x-pkcs12\": false,\n\t\"application/x-rar-compressed\": false,\n\t\"application/x-sh\": true,\n\t\"application/x-shockwave-flash\": false,\n\t\"application/x-stuffit\": false,\n\t\"application/x-tar\": true,\n\t\"application/x-virtualbox-hdd\": true,\n\t\"application/x-virtualbox-ova\": true,\n\t\"application/x-virtualbox-ovf\": true,\n\t\"application/x-virtualbox-vbox\": true,\n\t\"application/x-virtualbox-vbox-extpack\": false,\n\t\"application/x-virtualbox-vdi\": true,\n\t\"application/x-virtualbox-vhd\": true,\n\t\"application/x-virtualbox-vmdk\": true,\n\t\"application/x-web-app-manifest+json\": true,\n\t\"application/x-www-form-urlencoded\": true,\n\t\"application/x-xliff+xml\": true,\n\t\"application/x-xpinstall\": false,\n\t\"application/xacml+xml\": true,\n\t\"application/xaml+xml\": true,\n\t\"application/xcap-att+xml\": true,\n\t\"application/xcap-caps+xml\": true,\n\t\"application/xcap-diff+xml\": true,\n\t\"application/xcap-el+xml\": true,\n\t\"application/xcap-error+xml\": true,\n\t\"application/xcap-ns+xml\": true,\n\t\"application/xcon-conference-info+xml\": true,\n\t\"application/xcon-conference-info-diff+xml\": true,\n\t\"application/xenc+xml\": true,\n\t\"application/xhtml+xml\": true,\n\t\"application/xhtml-voice+xml\": true,\n\t\"application/xliff+xml\": true,\n\t\"application/xml\": true,\n\t\"application/xml-dtd\": true,\n\t\"application/xml-patch+xml\": true,\n\t\"application/xmpp+xml\": true,\n\t\"application/xop+xml\": true,\n\t\"application/xproc+xml\": true,\n\t\"application/xslt+xml\": true,\n\t\"application/xspf+xml\": true,\n\t\"application/xv+xml\": true,\n\t\"application/yang-data+json\": true,\n\t\"application/yang-data+xml\": true,\n\t\"application/yang-patch+json\": true,\n\t\"application/yang-patch+xml\": true,\n\t\"application/yin+xml\": true,\n\t\"application/zip\": false,\n\t\"audio/3gpp\": false,\n\t\"audio/basic\": false,\n\t\"audio/l24\": false,\n\t\"audio/mp3\": false,\n\t\"audio/mp4\": false,\n\t\"audio/mpeg\": false,\n\t\"audio/ogg\": false,\n\t\"audio/vnd.rn-realaudio\": false,\n\t\"audio/vnd.wave\": false,\n\t\"audio/vorbis\": false,\n\t\"audio/wav\": false,\n\t\"audio/wave\": false,\n\t\"audio/webm\": false,\n\t\"audio/x-aac\": false,\n\t\"audio/x-caf\": false,\n\t\"font/otf\": true,\n\t\"image/apng\": false,\n\t\"image/bmp\": true,\n\t\"image/gif\": false,\n\t\"image/jp2\": false,\n\t\"image/jpeg\": false,\n\t\"image/jpm\": false,\n\t\"image/jpx\": false,\n\t\"image/pjpeg\": false,\n\t\"image/png\": false,\n\t\"image/svg+xml\": true,\n\t\"image/tiff\": false,\n\t\"image/vnd.adobe.photoshop\": true,\n\t\"image/x-icon\": true,\n\t\"image/x-ms-bmp\": true,\n\t\"image/x-xcf\": false,\n\t\"message/http\": false,\n\t\"message/imdn+xml\": true,\n\t\"message/partial\": false,\n\t\"message/rfc822\": true,\n\t\"model/gltf+json\": true,\n\t\"model/gltf-binary\": true,\n\t\"model/iges\": false,\n\t\"model/mesh\": false,\n\t\"model/vnd.collada+xml\": true,\n\t\"model/vnd.moml+xml\": true,\n\t\"model/vrml\": false,\n\t\"model/x3d+binary\": false,\n\t\"model/x3d+vrml\": false,\n\t\"model/x3d+xml\": true,\n\t\"multipart/alternative\": false,\n\t\"multipart/encrypted\": false,\n\t\"multipart/form-data\": false,\n\t\"multipart/mixed\": false,\n\t\"multipart/related\": false,\n\t\"multipart/signed\": false,\n\t\"text/cache-manifest\": true,\n\t\"text/calender\": true,\n\t\"text/cmd\": true,\n\t\"text/css\": true,\n\t\"text/csv\": true,\n\t\"text/html\": true,\n\t\"text/javascript\": true,\n\t\"text/jsx\": true,\n\t\"text/markdown\": true,\n\t\"text/n3\": true,\n\t\"text/plain\": true,\n\t\"text/richtext\": true,\n\t\"text/rtf\": true,\n\t\"text/tab-separated-values\": true,\n\t\"text/uri-list\": true,\n\t\"text/vcard\": true,\n\t\"text/vtt\": true,\n\t\"text/x-gwt-rpc\": true,\n\t\"text/x-jquery-tmpl\": true,\n\t\"text/x-markdown\": true,\n\t\"text/x-org\": true,\n\t\"text/x-processing\": true,\n\t\"text/x-suse-ymp\": true,\n\t\"text/xml\": true,\n\t\"video/mp4\": false,\n\t\"video/mpeg\": false,\n\t\"video/ogg\": false,\n\t\"video/quicktime\": false,\n\t\"video/webm\": false,\n\t\"video/x-flv\": false,\n\t\"video/x-matroska\": false,\n\t\"video/x-ms-wmv\": false,\n\t\"x-shader/x-fragment\": true,\n\t\"x-shader/x-vertex\": true,\n}\n" }, { "path": "catfs/mio/compress/reader.go", "content": "package compress\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"sort\"\n\n\t\"github.com/sahib/brig/catfs/mio/chunkbuf\"\n)\n\n// Reader implements an decompressing reader\ntype Reader struct {\n\t// Underlying raw, compressed datastream.\n\trawR io.ReadSeeker\n\n\t// Index with records which contain chunk offsets.\n\tindex []record\n\n\t// Buffer holds currently read data; maxChunkSize.\n\tchunkBuf *chunkbuf.ChunkBuffer\n\n\t// Structure with parsed trailer.\n\ttrailer *trailer\n\n\t// Current seek offset in the compressed stream.\n\trawSeekOffset int64\n\n\t// Current seek offset in the uncompressed stream.\n\tzipSeekOffset int64\n\n\t// Marker to identify initial read.\n\tisInitialRead bool\n\n\t// Holds algorithm interface.\n\talgo Algorithm\n\n\t// buffer for reading in the raw stream for decoding.\n\trawBuf []byte\n\n\t// buffer to hold one chunks so Read() can take chunks of it.\n\tdecBuf []byte\n}\n\n// Seek implements io.Seeker\nfunc (r *Reader) Seek(destOff int64, whence int) (int64, error) {\n\tswitch whence {\n\tcase io.SeekEnd:\n\t\tif destOff > 0 {\n\t\t\treturn 0, io.EOF\n\t\t}\n\n\t\tif err := r.parseTrailerIfNeeded(); err != nil {\n\t\t\treturn 0, err\n\t\t}\n\n\t\treturn r.Seek(r.index[len(r.index)-1].rawOff+destOff, io.SeekStart)\n\tcase io.SeekCurrent:\n\t\treturn r.Seek(r.zipSeekOffset+destOff, io.SeekStart)\n\t}\n\n\tif err := r.parseTrailerIfNeeded(); err != nil {\n\t\treturn 0, err\n\t}\n\n\tif destOff < 0 {\n\t\treturn 0, io.EOF\n\t}\n\n\tdestRecord, _ := r.chunkLookup(destOff, true)\n\tcurrRecord, _ := r.chunkLookup(r.zipSeekOffset, true)\n\n\tr.rawSeekOffset = destRecord.zipOff\n\tr.zipSeekOffset = destOff\n\n\t// Don't re-read if offset is in current chunk.\n\tif currRecord.rawOff != destRecord.rawOff || !r.isInitialRead {\n\t\tif _, err := r.readZipChunk(); err != nil && err != io.EOF {\n\t\t\treturn 0, err\n\t\t}\n\t}\n\n\ttoRead := destOff - destRecord.rawOff\n\tif _, err := r.chunkBuf.Seek(toRead, io.SeekStart); err != nil {\n\t\treturn 0, err\n\t}\n\n\treturn destOff, nil\n}\n\n// Return start (prevRecord) and end (currRecord) of a chunk currOff is located\n// in. If currOff is 0, the first and second record is returned. If currOff is\n// at the end of file the end record (currRecord) is returned twice. The offset\n// difference (chunksize) between prevRecord and currRecord is then equal to 0.\nfunc (r *Reader) chunkLookup(currOff int64, isRawOff bool) (*record, *record) {\n\t// Get smallest index that is before given currOff.\n\ti := sort.Search(len(r.index), func(i int) bool {\n\t\tif isRawOff {\n\t\t\treturn r.index[i].rawOff > currOff\n\t\t}\n\t\treturn r.index[i].zipOff > currOff\n\t})\n\n\t// Beginning of the file, first chunk: prev offset is 0, curr offset is 1.\n\tif i == 0 {\n\t\treturn &r.index[i], &r.index[i+1]\n\t}\n\n\t// End of the file, last chunk: prev and curr offset is the last index.\n\tif i == len(r.index) {\n\t\treturn &r.index[i-1], &r.index[i-1]\n\t}\n\treturn &r.index[i-1], &r.index[i]\n}\n\nfunc (r *Reader) parseTrailerIfNeeded() error {\n\tif r.trailer != nil {\n\t\treturn nil\n\t}\n\n\t// Attempt to read the front header:\n\theaderBuf := [headerSize]byte{}\n\tif _, err := io.ReadFull(r.rawR, headerBuf[:]); err != nil {\n\t\treturn err\n\t}\n\n\theader, err := readHeader(headerBuf[:])\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Goto end of file and read trailer buffer.\n\tif _, err := r.rawR.Seek(-trailerSize, io.SeekEnd); err != nil {\n\t\treturn err\n\t}\n\n\tbuf := [trailerSize]byte{}\n\tn, err := io.ReadFull(r.rawR, buf[:])\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif n != trailerSize {\n\t\treturn fmt.Errorf(\"read trailer was too small: %d bytes\", n)\n\t}\n\n\tr.trailer = &trailer{}\n\tr.trailer.unmarshal(buf[:])\n\n\talgo, err := algorithmFromType(header.algo)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Allocate the rawBuf depending on the algorithm that was used.\n\t// Every compression algorithm might produce data that is bigger\n\t// than the original data in edge cases. `rawBuf` has to be big\n\t// enough to account for this edge case. We double check this\n\t// during Read() to avoid overflows.\n\tr.algo = algo\n\tr.rawBuf = make([]byte, algo.MaxEncodeBufferSize())\n\n\t// Seek and read index into buffer.\n\tseekIdx := -(int64(r.trailer.indexSize) + trailerSize)\n\tif _, err := r.rawR.Seek(seekIdx, io.SeekEnd); err != nil {\n\t\treturn err\n\t}\n\n\tindexBuf := make([]byte, r.trailer.indexSize)\n\tif _, err := io.ReadFull(r.rawR, indexBuf); err != nil {\n\t\treturn err\n\t}\n\n\t// Build index with records. A record encapsulates a raw offset and the\n\t// compressed offset it is mapped to.\n\tprevRecord := record{-1, -1}\n\tfor i := uint64(0); i < (r.trailer.indexSize / indexChunkSize); i++ {\n\t\tcurrRecord := record{}\n\t\tcurrRecord.unmarshal(indexBuf)\n\n\t\tif prevRecord.rawOff >= currRecord.rawOff {\n\t\t\treturn ErrBadIndex\n\t\t}\n\n\t\tif prevRecord.zipOff >= currRecord.zipOff {\n\t\t\treturn ErrBadIndex\n\t\t}\n\n\t\tr.index = append(r.index, currRecord)\n\t\tindexBuf = indexBuf[indexChunkSize:]\n\t}\n\n\t// Set Reader to beginning of file\n\tif _, err := r.rawR.Seek(headerSize, io.SeekStart); err != nil {\n\n\t\treturn err\n\t}\n\n\tr.rawSeekOffset = headerSize\n\tr.zipSeekOffset = 0\n\treturn nil\n}\n\n// WriteTo implements io.WriterTo\nfunc (r *Reader) WriteTo(w io.Writer) (int64, error) {\n\tif err := r.parseTrailerIfNeeded(); err != nil {\n\t\treturn 0, err\n\t}\n\n\twritten := int64(0)\n\n\tn, cerr := io.Copy(w, r.chunkBuf)\n\tif cerr != nil {\n\t\treturn n, cerr\n\t}\n\twritten += n\n\tfor {\n\t\tdecData, rerr := r.readZipChunk()\n\t\tif rerr == io.EOF {\n\t\t\treturn written, nil\n\t\t}\n\n\t\tif rerr != nil {\n\t\t\treturn written, rerr\n\t\t}\n\n\t\tn, werr := w.Write(decData)\n\t\twritten += int64(n)\n\n\t\tif werr != nil {\n\t\t\treturn written, werr\n\t\t}\n\t}\n}\n\n// Read reads len(p) bytes from the compressed stream into p.\nfunc (r *Reader) Read(p []byte) (int, error) {\n\tif err := r.parseTrailerIfNeeded(); err != nil {\n\t\treturn 0, err\n\t}\n\n\tread := 0\n\tfor {\n\t\tif r.chunkBuf.Len() != 0 {\n\t\t\tn, err := r.chunkBuf.Read(p)\n\n\t\t\t// NOTE: Read() might return io.EOF to indicate that the\n\t\t\t// chunk is exhausted. We should look at the next chunk\n\t\t\t// (readZipChunk will figure out if there are any)\n\t\t\tif err != nil && err != io.EOF {\n\t\t\t\treturn n, err\n\t\t\t}\n\n\t\t\tr.zipSeekOffset += int64(n)\n\t\t\tread += n\n\t\t\tp = p[n:]\n\t\t}\n\n\t\tif len(p) == 0 {\n\t\t\tbreak\n\t\t}\n\n\t\tif _, err := r.readZipChunk(); err != nil {\n\t\t\treturn read, err\n\t\t}\n\t}\n\n\treturn read, nil\n}\n\nfunc (r *Reader) fixZipChunk() (int64, error) {\n\t// Get the start and end record of the chunk currOff is located in.\n\tprevRecord, currRecord := r.chunkLookup(r.rawSeekOffset, false)\n\tif currRecord == nil || prevRecord == nil {\n\t\treturn 0, ErrBadIndex\n\t}\n\n\t// Determinate uncompressed chunksize; should only be 0 on empty file or at the end of file.\n\tchunkSize := currRecord.zipOff - prevRecord.zipOff\n\tif chunkSize == 0 {\n\t\treturn 0, io.EOF\n\t}\n\n\t// Set Reader to compressed offset.\n\tif _, err := r.rawR.Seek(prevRecord.zipOff, io.SeekStart); err != nil {\n\t\treturn 0, err\n\t}\n\n\tr.rawSeekOffset = currRecord.zipOff\n\tr.zipSeekOffset = prevRecord.rawOff\n\tr.isInitialRead = false\n\treturn chunkSize, nil\n}\n\nfunc (r *Reader) readZipChunk() ([]byte, error) {\n\t// Get current position of the Reader; offset of the compressed file.\n\tr.chunkBuf.Reset(nil)\n\tchunkSize, err := r.fixZipChunk()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif len(r.rawBuf) < int(chunkSize) {\n\t\t// NOTE: When len(r.rawBuf) is < chunkSize ErrShortBuffer is returned.\n\t\t// This should only happen on malicious input with far too high chunkSize.\n\t\t// r.rawBuf should be able to hold any compressed stream derived from any\n\t\t// block with size maxChunkSize.\n\t\treturn nil, io.ErrShortBuffer\n\t}\n\n\tn, err := io.ReadFull(r.rawR, r.rawBuf[:int(chunkSize)])\n\tif err != nil && err != io.EOF && err != io.ErrUnexpectedEOF {\n\t\treturn nil, err\n\t}\n\n\t// decData should be a slice of `r.decBuf` to avoid allocations.\n\tdecData, err := r.algo.Decode(r.decBuf, r.rawBuf[:n])\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tr.chunkBuf.Reset(decData)\n\treturn decData, nil\n}\n\n// NewReader returns a new ReadSeeker with compression support. As random access\n// is the purpose of this layer, a ReadSeeker is required as parameter. The used\n// compression algorithm is chosen based on trailer information.\nfunc NewReader(r io.ReadSeeker) *Reader {\n\treturn &Reader{\n\t\trawR: r,\n\t\tchunkBuf: chunkbuf.NewChunkBuffer([]byte{}),\n\t\tdecBuf: make([]byte, maxChunkSize),\n\t}\n}\n" }, { "path": "catfs/mio/compress/writer.go", "content": "package compress\n\nimport (\n\t\"bytes\"\n\t\"io\"\n\n\t\"github.com/sahib/brig/util\"\n)\n\n// Writer implements a compression writer.\ntype Writer struct {\n\t// Underlying raw, uncompressed data stream.\n\trawW io.Writer\n\n\t// Buffers data into maxChunkSize chunks.\n\tchunkBuf *bytes.Buffer\n\n\t// Index with records which contain chunk offsets.\n\tindex []record\n\n\t// Accumulator representing uncompressed offset.\n\trawOff int64\n\n\t// Accumulator representing compressed offset.\n\tzipOff int64\n\n\t// Holds trailer data.\n\ttrailer *trailer\n\n\t// Holds algorithm interface.\n\talgo Algorithm\n\n\t// Type of the algorithm\n\talgoType AlgorithmType\n\n\t// Becomes true after the first write.\n\theaderWritten bool\n\n\tencBuf []byte\n}\n\nfunc (w *Writer) addRecordToIndex() {\n\tw.index = append(w.index, record{w.rawOff, w.zipOff})\n}\n\nfunc (w *Writer) flushBuffer(data []byte) error {\n\tif len(data) <= 0 {\n\t\treturn nil\n\t}\n\n\t// Add record with start offset of the current chunk.\n\tw.addRecordToIndex()\n\n\t// Compress and flush the current chunk.\n\t// encData should be a slice of `w.encBuf`\n\tencData, err := w.algo.Encode(w.encBuf, data)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tn, err := w.rawW.Write(encData)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Update offset for the current chunk.\n\tw.rawOff += int64(len(data))\n\tw.zipOff += int64(n)\n\treturn nil\n}\n\nfunc (w *Writer) writeHeaderIfNeeded() error {\n\tif w.headerWritten {\n\t\treturn nil\n\t}\n\n\tif _, err := w.rawW.Write(makeHeader(w.algoType, currentVersion)); err != nil {\n\t\treturn err\n\t}\n\n\tw.headerWritten = true\n\tw.zipOff += headerSize\n\treturn nil\n}\n\n// ReadFrom implements io.ReaderFrom\nfunc (w *Writer) ReadFrom(r io.Reader) (n int64, err error) {\n\tread := 0\n\tbuf := make([]byte, maxChunkSize)\n\n\tif err := w.writeHeaderIfNeeded(); err != nil {\n\t\treturn 0, err\n\t}\n\n\tfor {\n\t\t// Only last block may be < maxChunkSize.\n\t\t// So we need to make sure to fill the buffer as best we can.\n\t\tn, rerr := io.ReadFull(r, buf)\n\t\tread += n\n\t\tif rerr != nil && rerr != io.ErrUnexpectedEOF && rerr != io.EOF {\n\t\t\treturn int64(read), rerr\n\t\t}\n\n\t\twerr := w.flushBuffer(buf[:n])\n\t\tif werr != nil && werr != io.EOF {\n\t\t\treturn int64(read), werr\n\t\t}\n\n\t\tif werr == io.EOF || rerr == io.EOF || rerr == io.ErrUnexpectedEOF {\n\t\t\treturn int64(read), nil\n\t\t}\n\t}\n}\n\nfunc (w *Writer) Write(p []byte) (n int, err error) {\n\tif err := w.writeHeaderIfNeeded(); err != nil {\n\t\treturn 0, err\n\t}\n\n\twritten := len(p)\n\t// Compress only maxChunkSize equal chunks.\n\tfor {\n\t\tn, _ := w.chunkBuf.Write(p[:util.Min(len(p), maxChunkSize)])\n\n\t\tif w.chunkBuf.Len() < maxChunkSize {\n\t\t\tbreak\n\t\t}\n\n\t\tif err := w.flushBuffer(w.chunkBuf.Next(maxChunkSize)); err != nil {\n\t\t\treturn 0, err\n\t\t}\n\t\tp = p[n:]\n\t}\n\treturn written, nil\n}\n\n// NewWriter returns a WriteCloser with compression support.\nfunc NewWriter(w io.Writer, algoType AlgorithmType) (*Writer, error) {\n\talgo, err := algorithmFromType(algoType)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &Writer{\n\t\trawW: w,\n\t\talgo: algo,\n\t\talgoType: algoType,\n\t\tencBuf: make([]byte, algo.MaxEncodeBufferSize()),\n\t\tchunkBuf: &bytes.Buffer{},\n\t\ttrailer: &trailer{},\n\t}, nil\n}\n\n// Close cleans up internal resources.\n// Make sure to call close always since it might write data.\nfunc (w *Writer) Close() error {\n\tif err := w.writeHeaderIfNeeded(); err != nil {\n\t\treturn err\n\t}\n\n\t// Write remaining bytes left in buffer and update index.\n\tif err := w.flushBuffer(w.chunkBuf.Bytes()); err != nil {\n\t\treturn err\n\t}\n\tw.addRecordToIndex()\n\n\t// Handle trailer of uncompressed file.\n\t// Write compression index trailer and close stream.\n\tw.trailer.indexSize = uint64(indexChunkSize * len(w.index))\n\tindexBuf := make([]byte, w.trailer.indexSize)\n\tindexBufStartOff := indexBuf\n\tfor _, record := range w.index {\n\t\trecord.marshal(indexBuf)\n\t\tindexBuf = indexBuf[indexChunkSize:]\n\t}\n\n\tif n, err := w.rawW.Write(indexBufStartOff); err != nil || uint64(n) != w.trailer.indexSize {\n\t\treturn err\n\t}\n\n\t// Write trailer buffer (algo, chunksize, indexsize)\n\t// at the end of file and close the stream.\n\ttrailerSizeBuf := make([]byte, trailerSize)\n\tw.trailer.marshal(trailerSizeBuf)\n\n\tif _, err := w.rawW.Write(trailerSizeBuf); err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n" }, { "path": "catfs/mio/doc.go", "content": "// Package mio (short for memory input/output) implements the layered io stack\n// of brig. This includes currently three major parts:\n//\n// - encrypt - Encryption and Decryption layer with seeking support.\n// - compress - Seekable Compression and Decompression with exchangable algorithms.\n// - overlay - In-Memory write overlay over a io.Reader with seek support.\n//\n// This package itself contains utils that stack those on top of each of other\n// in an already usable fashion.\npackage mio\n" }, { "path": "catfs/mio/encrypt/format.go", "content": "// Package encrypt implements the encryption layer of brig.\n// The file format used looks something like this:\n//\n// [HEADER][[BLOCKHEADER][PAYLOAD]...]\n//\n// HEADER is 20+16 bytes big and contains the following fields:\n// - 8 Byte: Magic number (to identify non-brig files quickly)\n// - 4 Byte: Flags (describing the stream)\n// - 2 Byte: Key length in bytes\n// - 2 Byte: Reserved for future use.\n//\t - 4 Byte: Maximum size of each block (last may be less)\n// - 16 Byte: MAC protecting the header from forgery\n//\n// BLOCKHEADER contains the following fields:\n// - 8 Byte: Nonce: Derived from the current block number.\n// The block number is checked to be correct on decryption.\n//\n// PAYLOAD contains the actual encrypted data, which includes a MAC at the end.\n// The size of the MAC depends on the algorithm, for poly1305 it's 16 bytes.\n//\n// All header metadata is encoded in little endian.\n//\n// Reader/Writer are capable or reading/writing this format. Additionally,\n// Reader supports efficient seeking into the encrypted data, provided the\n// underlying datastream supports seeking. SEEK_END is only supported when the\n// number of encrypted blocks is present in the header.\npackage encrypt\n\nimport (\n\t\"bytes\"\n\t\"crypto/aes\"\n\t\"crypto/cipher\"\n\t\"crypto/hmac\"\n\t\"encoding/binary\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\n\tchacha \"golang.org/x/crypto/chacha20poly1305\"\n\t\"golang.org/x/crypto/sha3\"\n)\n\n// Flags indicate with what options a stream was encoded.\n// Some flags are not compatible to each other, see below.\ntype Flags int32\n\n// Possible ciphers in Counter mode:\nconst (\n\t// FlagEmpty is invalid\n\tFlagEmpty = Flags(0)\n\n\t// FlagEncryptAES256GCM indicates the stream was encrypted with AES256 in GCM mode.\n\t// This should be fast on modern CPUs.\n\tFlagEncryptAES256GCM = Flags(1) << iota\n\n\t// FlagEncryptChaCha20 incidate that the stream was encrypted with ChaCha20.\n\t// This can be a good choice if your CPU does not support the AES-NI instruction set.\n\tFlagEncryptChaCha20\n\n\t// reserve some flags for more encryption types.\n\t// no particular reason, just want to have enc-type flags to be in line.\n\tflagReserved1\n\tflagReserved2\n\tflagReserved3\n\tflagReserved4\n\tflagReserved5\n\tflagReserved6\n\n\t// FlagCompressedInside indicates that the encrypted data was also compressed.\n\t// This can be used to decide at runtime what streaming is needed.\n\tFlagCompressedInside\n)\n\n// Other constants:\nconst (\n\t// Size of the header mac:\n\tmacSize = 16\n\n\t// current file format version, increment on incompatible changes.\n\tversion = 1\n\n\t// Size of the initial header:\n\theaderSize = 20 + macSize\n\n\t// Default maxBlockSize if not set\n\tdefaultMaxBlockSize = 64 * 1024\n\n\tdefaultDecBufferSize = defaultMaxBlockSize\n\tdefaultEncBufferSize = defaultMaxBlockSize + 40\n)\n\nvar (\n\t// MagicNumber contains the first 8 byte of every brig header.\n\t// For various reasons, it is the ascii string \"moosecat\".\n\tMagicNumber = []byte{\n\t\t0x6d, 0x6f, 0x6f, 0x73,\n\t\t0x65, 0x63, 0x61, 0x74,\n\t}\n)\n\n////////////////////\n// Header Parsing //\n////////////////////\n\n// GenerateHeader creates a valid header for the format file\nfunc GenerateHeader(key []byte, maxBlockSize int64, flags Flags) []byte {\n\t// This is in big endian:\n\theader := []byte{\n\t\t// magic number (8 Byte):\n\t\t0, 0, 0, 0, 0, 0, 0, 0,\n\t\t// Flags (4 byte):\n\t\t0, 0, 0, 0,\n\t\t// Key length (4 Byte):\n\t\t0, 0, 0, 0,\n\t\t// Block length (4 Byte):\n\t\t0, 0, 0, 0,\n\t\t// MAC Header (16 Byte):\n\t\t0, 0, 0, 0, 0, 0, 0, 0,\n\t\t0, 0, 0, 0, 0, 0, 0, 0,\n\t}\n\n\t// Magic number:\n\tcopy(header[:len(MagicNumber)], MagicNumber)\n\tbinary.LittleEndian.PutUint32(header[8:12], uint32(flags))\n\n\t// Encode key size (static at the moment):\n\tbinary.LittleEndian.PutUint32(header[12:16], uint32(32))\n\n\t// Encode max block size:\n\tbinary.LittleEndian.PutUint32(header[16:20], uint32(maxBlockSize))\n\n\t// Calculate a MAC of the header; this needs to be done last:\n\theaderMac := hmac.New(sha3.New224, key)\n\tif _, err := headerMac.Write(header[:headerSize-macSize]); err != nil {\n\t\treturn nil\n\t}\n\n\t// Copy the MAC to the output:\n\tshortHeaderMac := headerMac.Sum(nil)[:macSize]\n\tcopy(header[headerSize-macSize:headerSize], shortHeaderMac)\n\n\treturn header\n}\n\n// HeaderInfo represents a parsed header.\ntype HeaderInfo struct {\n\t// Version of the file format. Currently always 1.\n\tVersion uint16\n\n\t// Cipher type used in the file.\n\tCipherBit Flags\n\n\t// KeyLen is the number of bytes in the encryption key.\n\tKeyLen uint32\n\n\t// BlockLen is the max. number of bytes in a block.\n\t// The last block may be smaller.\n\tBlockLen uint32\n\n\t// Flags control the encryption algorithm and other things.\n\tFlags Flags\n}\n\nvar (\n\t// ErrSmallHeader is returned when the header is too small to parse.\n\t// Usually happens when trying to decrypt a raw stream.\n\tErrSmallHeader = errors.New(\"header is too small\")\n\n\t// ErrBadMagic is returned when the stream does not start with the magic number.\n\t// Usually happens when trying to decrypt a raw or compressed stream.\n\tErrBadMagic = errors.New(\"magic number missing\")\n\n\t// ErrBadFlags means that you passed an invalid flags combination\n\t// or the stream was modified to have wrong flags.\n\tErrBadFlags = errors.New(\"inconsistent header flags\")\n\n\t// ErrBadHeaderMAC means that the header is not what the writer originally\n\t// put into the stream. Usually means somebody or something changed it.\n\tErrBadHeaderMAC = errors.New(\"header mac differs from expected\")\n)\n\nfunc cipherTypeBitFromFlags(flags Flags) (Flags, error) {\n\tvar cipherBit Flags\n\tvar bits = []Flags{\n\t\tFlagEncryptAES256GCM,\n\t\tFlagEncryptChaCha20,\n\t}\n\n\tfor _, bit := range bits {\n\t\tif flags&bit == 0 {\n\t\t\tcontinue\n\t\t}\n\n\t\tif cipherBit != 0 {\n\t\t\t// only one bit at the same time allowed.\n\t\t\treturn 0, ErrBadFlags\n\t\t}\n\n\t\tcipherBit = bit\n\t}\n\n\tif cipherBit == 0 {\n\t\t// no algorithm set: also error out.\n\t\treturn 0, ErrBadFlags\n\t}\n\n\treturn cipherBit, nil\n}\n\n// ParseHeader parses the header of the format file. Returns the flags, key\n// and block length. If parsing fails, an error is returned.\nfunc ParseHeader(header, key []byte) (*HeaderInfo, error) {\n\tif len(header) < len(MagicNumber) {\n\t\treturn nil, ErrSmallHeader\n\t}\n\n\tif bytes.Compare(header[:len(MagicNumber)], MagicNumber) != 0 {\n\t\treturn nil, ErrBadMagic\n\t}\n\n\tif len(header) < headerSize {\n\t\treturn nil, ErrSmallHeader\n\t}\n\n\tflags := Flags(binary.LittleEndian.Uint32(header[8:12]))\n\tkeyLen := binary.LittleEndian.Uint32(header[12:16])\n\tblockLen := binary.LittleEndian.Uint32(header[16:20])\n\n\tcipherBit, err := cipherTypeBitFromFlags(flags)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Check the header mac:\n\theaderMac := hmac.New(sha3.New224, key)\n\tif _, err := headerMac.Write(header[:headerSize-macSize]); err != nil {\n\t\treturn nil, err\n\t}\n\n\tstoredMac := header[headerSize-macSize : headerSize]\n\tshortHeaderMac := headerMac.Sum(nil)[:macSize]\n\tif !hmac.Equal(shortHeaderMac, storedMac) {\n\t\treturn nil, ErrBadHeaderMAC\n\t}\n\n\treturn &HeaderInfo{\n\t\tVersion: version,\n\t\tCipherBit: cipherBit,\n\t\tKeyLen: keyLen,\n\t\tBlockLen: blockLen,\n\t\tFlags: flags,\n\t}, nil\n}\n\n//////////////////////\n// Common Utilities //\n//////////////////////\n\nfunc createAEADWorker(cipherType Flags, key []byte) (cipher.AEAD, error) {\n\tswitch cipherType {\n\tcase FlagEncryptAES256GCM:\n\t\tblock, err := aes.NewCipher(key)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\treturn cipher.NewGCM(block)\n\tcase FlagEncryptChaCha20:\n\t\treturn chacha.New(key)\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"no such cipher type: %d\", cipherType)\n\t}\n\n}\n\ntype aeadCommon struct {\n\t// Nonce that form the first aead.NonceSize() bytes\n\t// of the output\n\tnonce []byte\n\n\t// Key used for encryption/decryption\n\tkey []byte\n\n\t// For more information, see:\n\t// https://en.wikipedia.org/wiki/Authenticated_encryption\n\taead cipher.AEAD\n\n\t// Buffer for encrypted data (maxBlockSize + overhead)\n\tencBuf []byte\n}\n\nfunc (c *aeadCommon) initAeadCommon(key []byte, cipherBit Flags, maxBlockSize int64) error {\n\taead, err := createAEADWorker(cipherBit, key)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tc.encBuf = make([]byte, 0, maxBlockSize+int64(aead.Overhead()))\n\tc.nonce = make([]byte, aead.NonceSize())\n\tc.aead = aead\n\tc.key = key\n\treturn nil\n}\n\n// Encrypt is a utility function which encrypts the data from source with key\n// and writes the resulting encrypted data to dest.\nfunc Encrypt(key []byte, source io.Reader, dest io.Writer, flags Flags) (int64, error) {\n\tlayer, err := NewWriter(dest, key, flags)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\tn, err := io.CopyBuffer(layer, source, make([]byte, defaultEncBufferSize))\n\tif err != nil {\n\t\treturn n, err\n\t}\n\n\treturn n, layer.Close()\n}\n\n// Decrypt is a utility function which decrypts the data from source with key\n// and writes the resulting encrypted data to dest.\nfunc Decrypt(key []byte, source io.Reader, dest io.Writer) (int64, error) {\n\tlayer, err := NewReader(source, key)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\treturn io.CopyBuffer(dest, layer, make([]byte, defaultDecBufferSize))\n}\n" }, { "path": "catfs/mio/encrypt/format_test.go", "content": "package encrypt\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"math\"\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/sahib/brig/util\"\n\t\"github.com/sahib/brig/util/testutil\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nvar TestKey = []byte(\"01234567890ABCDE01234567890ABCDE\")\n\nconst ExtraDebug = false\n\nfunc openFiles(from, to string) (*os.File, *os.File, error) {\n\tfdFrom, err := os.Open(from)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\tfdTo, err := os.OpenFile(to, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0755)\n\tif err != nil {\n\t\tfdFrom.Close()\n\t\treturn nil, nil, err\n\t}\n\n\treturn fdFrom, fdTo, nil\n}\n\nfunc encryptFile(key []byte, from, to string) (n int64, outErr error) {\n\tfdFrom, fdTo, err := openFiles(from, to)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\tdefer func() {\n\t\t// Only fdTo needs to be closed, Decrypt closes fdFrom.\n\t\tif err := fdFrom.Close(); err != nil {\n\t\t\toutErr = err\n\t\t}\n\t\tif err := fdTo.Close(); err != nil {\n\t\t\toutErr = err\n\t\t}\n\t}()\n\n\treturn Encrypt(key, fdFrom, fdTo, FlagEncryptAES256GCM)\n}\n\nfunc decryptFile(key []byte, from, to string) (n int64, outErr error) {\n\tfdFrom, fdTo, err := openFiles(from, to)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\tdefer func() {\n\t\tif err := fdTo.Close(); err != nil {\n\t\t\toutErr = err\n\t\t\treturn\n\t\t}\n\n\t\tif err := fdFrom.Close(); err != nil {\n\t\t\toutErr = err\n\t\t\treturn\n\t\t}\n\t}()\n\n\treturn Decrypt(key, fdFrom, fdTo)\n}\n\nfunc remover(t *testing.T, path string) {\n\tif err := os.Remove(path); err != nil {\n\t\tt.Errorf(\"Could not remove temp file: %v\", err)\n\t}\n}\n\nfunc testSimpleEncDec(t *testing.T, size int64) {\n\tpath := testutil.CreateFile(int64(size))\n\tdefer remover(t, path)\n\n\tencPath := path + \"_enc\"\n\tdecPath := path + \"_dec\"\n\n\t_, err := encryptFile(TestKey, path, encPath)\n\tdefer remover(t, encPath)\n\n\tif err != nil {\n\t\tt.Errorf(\"Encrypt failed: %v\", err)\n\t}\n\n\t_, err = decryptFile(TestKey, encPath, decPath)\n\tdefer remover(t, decPath)\n\n\tif err != nil {\n\t\tt.Errorf(\"Decrypt failed: %v\", err)\n\t}\n\n\ta, _ := ioutil.ReadFile(path)\n\tb, _ := ioutil.ReadFile(decPath)\n\tc, _ := ioutil.ReadFile(encPath)\n\n\tif !bytes.Equal(a, b) {\n\t\tt.Errorf(\"Source and decrypted not equal\")\n\t}\n\n\tif bytes.Equal(a, c) && size != 0 {\n\t\tt.Errorf(\"Source was not encrypted (same as source)\")\n\t\tt.Errorf(\"%v|||%v|||%v\", a, b, c)\n\t}\n}\n\nfunc TestSimpleEncDec(t *testing.T) {\n\tt.Parallel()\n\n\tfor _, size := range SizeTests {\n\t\tif ExtraDebug {\n\t\t\tt.Logf(\"Testing SimpleEncDec for size %d\", size)\n\t\t}\n\n\t\tt.Run(fmt.Sprintf(\"size-%d\", size), func(t *testing.T) {\n\t\t\ttestSimpleEncDec(t, size)\n\t\t})\n\t}\n}\n\nvar SizeTests = []int64{\n\t0,\n\t1,\n\tdefaultMaxBlockSize - 1,\n\tdefaultMaxBlockSize,\n\tdefaultMaxBlockSize + 10,\n\tdefaultDecBufferSize - 1,\n\tdefaultDecBufferSize,\n\tdefaultDecBufferSize + 1,\n\tdefaultEncBufferSize - 1,\n\tdefaultEncBufferSize,\n\t7 * defaultEncBufferSize,\n\t7*defaultEncBufferSize - 1,\n\tdefaultEncBufferSize + 1,\n}\n\ntype seekTest struct {\n\tWhence int\n\tOffset float64\n\tError error\n}\n\nvar SeekTests = []seekTest{\n\t// Jump to the mid:\n\t{io.SeekStart, 0.5, nil},\n\t// Should stay the same:\n\t{io.SeekCurrent, 0, nil},\n\t// Jump a quarter forth:\n\t{io.SeekCurrent, 0.25, nil},\n\t// Jump a half back:\n\t{io.SeekCurrent, -0.5, nil},\n\t// Jump back to the half:\n\t{io.SeekCurrent, 0.25, nil},\n\t// See if SEEK_END works:\n\t{io.SeekEnd, -0.5, nil},\n\t// This triggered a crash earlier:\n\t{io.SeekEnd, -2, io.EOF},\n\t// Im guessing now:\n\t{io.SeekEnd, -1.0 / 4096, nil},\n}\n\nfunc BenchmarkEncDec(b *testing.B) {\n\tfor n := 0; n < b.N; n++ {\n\t\ttestSimpleEncDec(nil, defaultMaxBlockSize*100)\n\t}\n}\n\nfunc TestSeek(t *testing.T) {\n\tfor _, size := range SizeTests {\n\t\ttestSeek(t, int64(size), false, false)\n\t\ttestSeek(t, int64(size), false, true)\n\t\ttestSeek(t, int64(size), true, false)\n\t\ttestSeek(t, int64(size), true, true)\n\n\t\tif t.Failed() {\n\t\t\tbreak\n\t\t}\n\t}\n}\n\nfunc testSeek(t *testing.T, N int64, readFrom, writeTo bool) {\n\tsourceData := testutil.CreateDummyBuf(N)\n\tsource := bytes.NewBuffer(sourceData)\n\tshared := &bytes.Buffer{}\n\n\tif ExtraDebug {\n\t\tt.Logf(\"Testing seek for size %d\", N)\n\t}\n\n\tenc, err := NewWriter(shared, TestKey, FlagEncryptAES256GCM)\n\tif err != nil {\n\t\tt.Errorf(\"Creating an encrypted writer failed: %v\", err)\n\t\treturn\n\t}\n\n\t// Encrypt:\n\tif _, err = testutil.DumbCopy(enc, source, readFrom, writeTo); err != nil {\n\t\tt.Errorf(\"copy(enc, source) failed %v\", err)\n\t\treturn\n\t}\n\n\t// This needs to be here, since close writes\n\t// left over data to the write stream\n\tif err = enc.Close(); err != nil {\n\t\tt.Errorf(\"close(enc): %v\", err)\n\t\treturn\n\t}\n\n\tsharedReader := bytes.NewReader(shared.Bytes())\n\tdecLayer, err := NewReader(sharedReader, TestKey)\n\tif err != nil {\n\t\tt.Errorf(\"creating new reader failed: %v\", err)\n\t\treturn\n\t}\n\n\tlastJump := int64(0)\n\n\tfor _, test := range SeekTests {\n\t\tlastJump = testSeekOneWhence(\n\t\t\tt, N, readFrom, writeTo, lastJump, test, decLayer, sourceData,\n\t\t)\n\t}\n}\n\nfunc testSeekOneWhence(\n\tt *testing.T, N int64, readFrom, writeTo bool,\n\tlastJump int64, test seekTest,\n\tdecLayer *Reader, sourceData []byte,\n) int64 {\n\trealOffset := int64(math.Floor(.5 + test.Offset*float64(N)))\n\n\twhence := map[int]string{\n\t\t0: \"SEEK_SET\",\n\t\t1: \"SEEK_CUR\",\n\t\t2: \"SEEK_END\",\n\t}[test.Whence]\n\n\texptOffset := int64(0)\n\tswitch test.Whence {\n\tcase io.SeekStart:\n\t\texptOffset = realOffset\n\tcase io.SeekCurrent:\n\t\texptOffset = lastJump + realOffset\n\tcase io.SeekEnd:\n\t\texptOffset = N + realOffset\n\tdefault:\n\t\tpanic(\"Bad whence\")\n\t}\n\n\tif ExtraDebug {\n\t\tt.Logf(\n\t\t\t\" => Seek(%v, %v) -> %v (size: %v)\",\n\t\t\trealOffset,\n\t\t\twhence,\n\t\t\texptOffset,\n\t\t\tN,\n\t\t)\n\t}\n\n\tjumpedTo, err := decLayer.Seek(realOffset, test.Whence)\n\tif err != test.Error {\n\t\tif err != io.EOF && N != 0 {\n\t\t\tt.Fatalf(\n\t\t\t\t\"Seek(%v, %v) produced an error: %v (should be %v)\",\n\t\t\t\trealOffset,\n\t\t\t\twhence,\n\t\t\t\terr,\n\t\t\t\ttest.Error,\n\t\t\t)\n\t\t}\n\t}\n\n\tif test.Error != nil {\n\t\treturn lastJump\n\t}\n\n\tif jumpedTo != exptOffset {\n\t\tt.Errorf(\n\t\t\t\"Seek(%v, %v) jumped badly. Should be %v, was %v\",\n\t\t\trealOffset,\n\t\t\twhence,\n\t\t\texptOffset,\n\t\t\tjumpedTo,\n\t\t)\n\t\treturn -1\n\t}\n\n\t// Decrypt and check if the contents are okay:\n\tdest := bytes.NewBuffer(nil)\n\n\tcopiedBytes, err := testutil.DumbCopy(dest, decLayer, readFrom, writeTo)\n\tif err != nil {\n\t\tt.Errorf(\"Decrypt failed: %v\", err)\n\t\treturn jumpedTo\n\t}\n\n\tif copiedBytes != N-jumpedTo {\n\t\tt.Errorf(\"Copied different amount of decrypted data than expected.\")\n\t\tt.Errorf(\"Should be %v, was %v bytes.\", N-jumpedTo, copiedBytes)\n\t}\n\n\t// Check the data actually matches the source data.\n\tif !bytes.Equal(sourceData[jumpedTo:], dest.Bytes()) {\n\t\tt.Errorf(\"Seeked data does not match expectations.\")\n\t\tt.Errorf(\"\\tEXPECTED: %v\", util.OmitBytes(sourceData[jumpedTo:], 10))\n\t\tt.Errorf(\"\\tGOT: %v\", util.OmitBytes(dest.Bytes(), 10))\n\t\treturn jumpedTo\n\t}\n\n\t// Jump back, so the other tests continue to work:\n\tjumpedAgain, err := decLayer.Seek(jumpedTo, io.SeekStart)\n\tif err != nil {\n\t\tt.Errorf(\"Seeking not possible after reading: %v\", err)\n\t\treturn jumpedTo\n\t}\n\n\tif jumpedTo != jumpedAgain {\n\t\tt.Errorf(\"Jumping back to original pos failed.\")\n\t\tt.Errorf(\"Should be %v, was %v.\", jumpedTo, jumpedAgain)\n\t\treturn jumpedTo\n\t}\n\n\treturn jumpedTo\n}\n\nfunc TestEmptyFile(t *testing.T) {\n\tsrcBuf := []byte{}\n\tdstBuf := []byte{}\n\ttmpBuf := &bytes.Buffer{}\n\n\tsrc := bytes.NewReader(srcBuf)\n\tdst := bytes.NewBuffer(dstBuf)\n\n\tenc, err := NewWriter(tmpBuf, TestKey, FlagEncryptAES256GCM)\n\tif err != nil {\n\t\tt.Errorf(\"TestEmpyFile: creating writer failed: %v\", err)\n\t\treturn\n\t}\n\n\tif _, err = io.Copy(enc, src); err != nil {\n\t\tt.Errorf(\"TestEmpyFile: copy(enc, src) failed: %v\", err)\n\t\treturn\n\t}\n\n\tif err = enc.Close(); err != nil {\n\t\tt.Errorf(\"TestEmpyFile: close(enc) failed: %v\", err)\n\t\treturn\n\t}\n\n\tdec, err := NewReader(bytes.NewReader(tmpBuf.Bytes()), TestKey)\n\tif err != nil {\n\t\tt.Errorf(\"TestEmpyFile: creating reader failed: %v\", err)\n\t\treturn\n\t}\n\n\tif _, err = dec.Seek(10, io.SeekStart); err != io.EOF {\n\t\tt.Errorf(\"Seek failed: %v\", err)\n\t\treturn\n\t}\n\n\tif _, err = io.Copy(dst, dec); err != nil {\n\t\tt.Errorf(\"TestEmpyFile: copy(dst, dec) failed: %v\", err)\n\t\treturn\n\t}\n\n\tif !bytes.Equal(srcBuf, dstBuf) {\n\t\tt.Errorf(\"TestEmpyFile: Not empty: src=%v dst=%v\", srcBuf, dstBuf)\n\t\treturn\n\t}\n}\n\n// Test if encrypting the same plaintext twice yields\n// the same ciphertext. This is a crucial property for brig, although it\n// has some security implications (i.e. no real random etc.)\nfunc TestEncryptedTheSame(t *testing.T) {\n\tsourceData := testutil.CreateDummyBuf(3 * defaultMaxBlockSize)\n\tencOne := &bytes.Buffer{}\n\tencTwo := &bytes.Buffer{}\n\n\tn1, err := Encrypt(\n\t\tTestKey,\n\t\tbytes.NewReader(sourceData),\n\t\tencOne,\n\t\tFlagEncryptAES256GCM,\n\t)\n\tif err != nil {\n\t\tt.Errorf(\"TestEncryptedTheSame: Encrypting first failed: %v\", err)\n\t\treturn\n\t}\n\n\tn2, err := Encrypt(\n\t\tTestKey,\n\t\tbytes.NewReader(sourceData),\n\t\tencTwo,\n\t\tFlagEncryptAES256GCM,\n\t)\n\tif err != nil {\n\t\tt.Errorf(\"TestEncryptedTheSame: Encrypting second failed: %v\", err)\n\t\treturn\n\t}\n\n\tif n1 != n2 {\n\t\tt.Errorf(\"TestEncryptedTheSame: Ciphertexts differ in length.\")\n\t\treturn\n\t}\n\n\tif !bytes.Equal(encOne.Bytes(), encTwo.Bytes()) {\n\t\tt.Errorf(\"TestEncryptedTheSame: Ciphertext differ, you failed.\")\n\t\tt.Errorf(\"\\tOne: %v\", encOne.Bytes())\n\t\tt.Errorf(\"\\tTwo: %v\", encTwo.Bytes())\n\t\treturn\n\t}\n}\n\n// Test if swapping small parts of the output\nfunc TestEncryptedByteSwaps(t *testing.T) {\n\tdata1 := testutil.CreateDummyBuf(2 * defaultMaxBlockSize)\n\tdata2 := testutil.CreateDummyBuf(2 * defaultMaxBlockSize)\n\tdata3 := testutil.CreateDummyBuf(2 * defaultMaxBlockSize)\n\n\t// Do a small modification in the beginning.\n\tdata2[0]++\n\n\t// Do a small modification in the end.\n\tdata3[2*defaultMaxBlockSize-1]++\n\n\t// Encrypt all data samples:\n\tencBuf1 := &bytes.Buffer{}\n\tencBuf2 := &bytes.Buffer{}\n\tencBuf3 := &bytes.Buffer{}\n\n\tvar err error\n\t_, err = Encrypt(\n\t\tTestKey,\n\t\tbytes.NewReader(data1),\n\t\tencBuf1,\n\t\tFlagEncryptChaCha20,\n\t)\n\trequire.Nil(t, err)\n\n\t_, err = Encrypt(\n\t\tTestKey,\n\t\tbytes.NewReader(data2),\n\t\tencBuf2,\n\t\tFlagEncryptChaCha20,\n\t)\n\trequire.Nil(t, err)\n\n\t_, err = Encrypt(\n\t\tTestKey,\n\t\tbytes.NewReader(data3),\n\t\tencBuf3,\n\t\tFlagEncryptChaCha20,\n\t)\n\trequire.Nil(t, err)\n\n\tencData1 := encBuf1.Bytes()\n\tencData2 := encBuf2.Bytes()\n\tencData3 := encBuf3.Bytes()\n\n\t// It should be all the same with a one-byte change.\n\trequire.Equal(t, len(encData1), len(encData2))\n\trequire.Equal(t, len(encData2), len(encData3))\n\n\t// s = full size; m = start of second block\n\ts := len(encData1)\n\tm := len(encData1)/2 + headerSize\n\n\t// Require that only the first block was tainted, other block should be same.\n\trequire.False(t, bytes.Equal(encData1[0:m], encData2[0:m]))\n\trequire.True(t, bytes.Equal(encData1[m:s], encData2[m:s]))\n\n\t// Require that the last block was tainted, first block should be same\n\trequire.True(t, bytes.Equal(encData1[0:m], encData3[0:m]))\n\trequire.False(t, bytes.Equal(encData1[m:s], encData3[m:s]))\n}\n" }, { "path": "catfs/mio/encrypt/reader.go", "content": "package encrypt\n\nimport (\n\t\"bytes\"\n\t\"encoding/binary\"\n\t\"fmt\"\n\t\"io\"\n)\n\n// Reader decrypts and encrypted stream from Reader.\ntype Reader struct {\n\t// Underlying reader\n\tio.Reader\n\n\taeadCommon\n\n\t// Caches leftovers from unread blocks\n\tbacklog *bytes.Reader\n\n\t// Last index of the byte the user visited.\n\t// (Used to avoid re-reads in Seek())\n\t// This does *not* equal the seek offset of the underlying stream.\n\tlastDecSeekPos int64\n\n\t// lastEncSeekPos saves the current position of the underlying stream.\n\t// it is used mostly for ensuring SEEK_END works.\n\tlastEncSeekPos int64\n\n\t// Parsed header info\n\tinfo *HeaderInfo\n\n\t// true once readHeader() was called\n\tparsedHeader bool\n\n\t// Buffer for decrypted data (MaxBlockSize big)\n\tdecBuf []byte\n\n\t// true as long readBlock was not successful\n\tisInitialRead bool\n\n\t// Total size of the underlying stream in bytes.\n\t// This is only set when SEEK_END was used.\n\tendOffsetEnc int64\n}\n\nfunc (r *Reader) readHeaderIfNotDone() error {\n\tif r.parsedHeader {\n\t\treturn nil\n\t}\n\n\tr.parsedHeader = true\n\n\theader := make([]byte, headerSize)\n\tn, err := io.ReadFull(r.Reader, header)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif n != headerSize {\n\t\treturn fmt.Errorf(\"No valid header found, damaged file?\")\n\t}\n\n\tinfo, err := ParseHeader(header, r.key)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif info.Version != 1 {\n\t\treturn fmt.Errorf(\"this implementation does not support versions != 1\")\n\t}\n\n\tif uint32(len(r.key)) != info.KeyLen {\n\t\treturn fmt.Errorf(\n\t\t\t\"key length differs: file=%d, user=%d\",\n\t\t\tinfo.KeyLen,\n\t\t\tlen(r.key),\n\t\t)\n\t}\n\n\tr.info = info\n\tif err := r.initAeadCommon(\n\t\tr.key,\n\t\tinfo.CipherBit,\n\t\tint64(r.info.BlockLen),\n\t); err != nil {\n\t\treturn err\n\t}\n\n\tr.lastEncSeekPos += headerSize\n\tr.decBuf = make([]byte, 0, r.info.BlockLen)\n\treturn nil\n}\n\n// Flags will return the flags stored in the header of the encrypted stream.\n// If the header was not read yet, it will attempt to read it.\nfunc (r *Reader) Flags() (Flags, error) {\n\t// Make sure we have the info needed to parse the header:\n\tif err := r.readHeaderIfNotDone(); err != nil {\n\t\treturn 0, err\n\t}\n\n\treturn r.info.Flags, nil\n}\n\n// Read from source and decrypt.\n//\n// This method always decrypts one block to optimize for continuous reads. If\n// dest is too small to hold the block, the decrypted text is cached for the\n// next read.\nfunc (r *Reader) Read(dest []byte) (int, error) {\n\t// Make sure we have the info needed to parse the header:\n\tif err := r.readHeaderIfNotDone(); err != nil {\n\t\treturn 0, err\n\t}\n\n\treadBytes := 0\n\n\t// Try our best to fill len(dest)\n\tfor readBytes < len(dest) {\n\t\tif r.backlog.Len() == 0 {\n\t\t\tif _, rerr := r.readBlock(); rerr != nil && rerr != io.EOF {\n\t\t\t\treturn readBytes, rerr\n\t\t\t}\n\t\t}\n\n\t\tn, berr := r.backlog.Read(dest[readBytes:])\n\t\tr.lastDecSeekPos += int64(n)\n\t\treadBytes += n\n\n\t\tif berr == io.EOF {\n\t\t\treturn readBytes, io.EOF\n\t\t}\n\t}\n\n\treturn readBytes, nil\n}\n\n// Fill internal buffer with current block\nfunc (r *Reader) readBlock() (int, error) {\n\tif r.info == nil {\n\t\treturn 0, fmt.Errorf(\"Invalid header data\")\n\t}\n\n\t// Read nonce:\n\tif n, err := io.ReadFull(r.Reader, r.nonce); err != nil {\n\t\treturn 0, err\n\t} else if n != r.aead.NonceSize() {\n\t\treturn 0, fmt.Errorf(\n\t\t\t\"nonce size mismatch; should: %d - have: %d (err: %v)\",\n\t\t\tr.aead.NonceSize(),\n\t\t\tn,\n\t\t\terr,\n\t\t)\n\t}\n\n\t// Convert to block number:\n\treadBlockNum := binary.LittleEndian.Uint64(r.nonce)\n\n\t// Check the block number:\n\tcurrBlockNum := uint64(r.lastDecSeekPos / int64(r.info.BlockLen))\n\tif currBlockNum != readBlockNum {\n\t\treturn 0, fmt.Errorf(\n\t\t\t\"bad block number; as %d, should be %d\", readBlockNum, currBlockNum,\n\t\t)\n\t}\n\n\t// Read the *whole* block from the raw stream\n\tN := int(r.info.BlockLen) + r.aead.Overhead()\n\tn, err := io.ReadAtLeast(r.Reader, r.encBuf[:N], N)\n\tif err != nil && err != io.EOF && err != io.ErrUnexpectedEOF {\n\t\treturn 0, err\n\t}\n\n\tr.lastEncSeekPos += int64(n) + int64(len(r.nonce))\n\n\tr.decBuf, err = r.aead.Open(r.decBuf[:0], r.nonce, r.encBuf[:n], nil)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\tr.backlog.Reset(r.decBuf)\n\tr.isInitialRead = false\n\n\treturn len(r.decBuf), nil\n}\n\n// Seek into the encrypted stream.\n//\n// Note that the seek offset is relative to the decrypted data,\n// not to the underlying, encrypted stream.\nfunc (r *Reader) Seek(offset int64, whence int) (int64, error) {\n\t// Check if seeking is supported:\n\tseeker, ok := r.Reader.(io.Seeker)\n\tif !ok {\n\t\treturn 0, fmt.Errorf(\"seek is not supported by underlying stream\")\n\t}\n\n\tif err := r.readHeaderIfNotDone(); err != nil {\n\t\treturn 0, err\n\t}\n\n\t// set to true when an actual call to seeker.Seek() was made.\n\twasMoved := false\n\n\t// Constants and assumption on the stream below:\n\tblockHeaderSize := int64(r.aead.NonceSize())\n\tblockOverhead := blockHeaderSize + int64(r.aead.Overhead())\n\ttotalBlockSize := blockOverhead + int64(r.info.BlockLen)\n\n\t// absolute Offset in the decrypted stream\n\tabsOffsetDec := int64(0)\n\n\t// Convert possibly relative offset to absolute offset:\n\tswitch whence {\n\tcase io.SeekCurrent:\n\t\tabsOffsetDec = r.lastDecSeekPos + offset\n\tcase io.SeekStart:\n\t\tabsOffsetDec = offset\n\tcase io.SeekEnd:\n\t\t// Try to figure out the end of the stream.\n\t\t// This might be inefficient for some underlying readers,\n\t\t// but is probably okay for ipfs.\n\t\t//\n\t\t// NOTE: We cache this not only for performance, but also\n\t\t// as a workaround for a bug in ipfs.\n\t\t// See: https://github.com/ipfs/go-ipfs/issues/2567\n\t\tif r.endOffsetEnc < 0 {\n\t\t\tendOffsetEnc, err := seeker.Seek(0, io.SeekEnd)\n\t\t\tif err != nil && err != io.EOF {\n\t\t\t\treturn 0, err\n\t\t\t}\n\n\t\t\tr.endOffsetEnc = endOffsetEnc\n\t\t}\n\n\t\t// This computation is verbose on purporse,\n\t\t// since the details might be confusing.\n\t\tencLen := (r.endOffsetEnc - headerSize)\n\t\tencRest := encLen % totalBlockSize\n\t\tdecBlocks := encLen / totalBlockSize\n\n\t\tendOffsetDec := decBlocks * int64(r.info.BlockLen)\n\t\tif encRest > 0 {\n\t\t\tendOffsetDec += encRest - blockOverhead\n\t\t}\n\t\tabsOffsetDec = endOffsetDec + offset\n\n\t\tif absOffsetDec < 0 {\n\t\t\t// That's the wrong end of file...\n\t\t\treturn 0, io.EOF\n\t\t}\n\n\t\t// For SEEK_END we need to make sure that we move the seek pointer\n\t\t// back to a sensible position when we decide that no actual move\n\t\t// is necessary further down this function.\n\t\tdefer func() {\n\t\t\tif !wasMoved {\n\t\t\t\tseeker.Seek(r.lastEncSeekPos, io.SeekStart)\n\t\t\t}\n\t\t}()\n\t}\n\n\tif absOffsetDec < 0 {\n\t\treturn 0, fmt.Errorf(\"Negative seek index: %d\", absOffsetDec)\n\t}\n\n\t// Caller wanted to know only the current stream pos:\n\tif absOffsetDec == r.lastDecSeekPos {\n\t\treturn absOffsetDec, nil\n\t}\n\n\t// Convert decrypted offset to encrypted offset\n\tabsOffsetEnc := headerSize + ((absOffsetDec / int64(r.info.BlockLen)) * totalBlockSize)\n\n\t// Check if we're still in the same block as last time:\n\tblockNum := absOffsetEnc / totalBlockSize\n\tlastBlockNum := r.lastDecSeekPos / int64(r.info.BlockLen)\n\n\tr.lastDecSeekPos = absOffsetDec\n\n\tif lastBlockNum != blockNum || r.isInitialRead || whence == io.SeekEnd {\n\t\tr.lastEncSeekPos = absOffsetEnc\n\n\t\t// Seek to the beginning of the encrypted block:\n\t\twasMoved = true\n\t\tif _, err := seeker.Seek(absOffsetEnc, io.SeekStart); err != nil {\n\t\t\treturn 0, err\n\t\t}\n\n\t\t// Make read consume the current block:\n\t\tif _, err := r.readBlock(); err != nil {\n\t\t\treturn 0, err\n\t\t}\n\t}\n\n\t// reslice the backlog, so Read() does not return skipped data.\n\tif _, err := r.backlog.Seek(\n\t\tabsOffsetDec%int64(r.info.BlockLen),\n\t\tio.SeekStart,\n\t); err != nil {\n\t\treturn 0, err\n\t}\n\n\treturn absOffsetDec, nil\n}\n\n// WriteTo copies all data from `r` to `w`.\n//\n// It is intended to avoid unneeded copying by choosing a suitable buffer size\n// and by directly reading block after block. io.Copy will use it automatically.\n//\n// It returns the number of written bytes and possible errors (but no io.EOF)\nfunc (r *Reader) WriteTo(w io.Writer) (int64, error) {\n\t// Make sure we have the info needed to parse the header:\n\tif err := r.readHeaderIfNotDone(); err != nil {\n\t\treturn 0, err\n\t}\n\n\tn := int64(0)\n\n\t// Backlog might be still filled if Read() or Seek() was done before:\n\tif r.backlog.Len() > 0 {\n\t\tbn, err := r.backlog.WriteTo(w)\n\t\tif err != nil {\n\t\t\treturn bn, err\n\t\t}\n\n\t\tn += bn\n\t\tr.lastDecSeekPos += bn\n\t}\n\n\tfor {\n\t\tnread, rerr := r.readBlock()\n\t\tif rerr != nil && rerr != io.EOF {\n\t\t\treturn n, rerr\n\t\t}\n\n\t\tr.lastDecSeekPos += int64(nread)\n\n\t\tnwrite, werr := w.Write(r.decBuf[:nread])\n\t\tif werr != nil {\n\t\t\treturn n, werr\n\t\t}\n\n\t\tn += int64(nwrite)\n\n\t\tif nwrite != nread {\n\t\t\treturn n, io.ErrShortWrite\n\t\t}\n\n\t\tif rerr == io.EOF {\n\t\t\tbreak\n\t\t}\n\t}\n\n\treturn n, nil\n}\n\n// NewReader creates a new encrypted reader and validates the file header.\n// The key is required to be KeySize bytes long.\nfunc NewReader(r io.Reader, key []byte) (*Reader, error) {\n\treader := &Reader{\n\t\tReader: r,\n\t\tbacklog: bytes.NewReader([]byte{}),\n\t\tparsedHeader: false,\n\t\tisInitialRead: true,\n\t\tendOffsetEnc: -1,\n\t\taeadCommon: aeadCommon{\n\t\t\tkey: key,\n\t\t},\n\t}\n\n\treturn reader, nil\n}\n" }, { "path": "catfs/mio/encrypt/writer.go", "content": "package encrypt\n\nimport (\n\t\"bytes\"\n\t\"encoding/binary\"\n\t\"errors\"\n\t\"io\"\n)\n\nvar (\n\t// ErrBadBlockSize is returned when the data is damaged and has an invalid block size\n\tErrBadBlockSize = errors.New(\"underlying reader failed to read full w.maxBlockSize\")\n\n\t// ErrMixedMethods is returned when calling Write() with ReadFrom() together.\n\tErrMixedMethods = errors.New(\"mixing Write() and ReadFrom() is not allowed\")\n)\n\n// Writer encrypts the data stream before writing to Writer.\ntype Writer struct {\n\t// Internal Writer we would write to.\n\tio.Writer\n\n\t// Common fields with Reader\n\taeadCommon\n\n\t// A buffer that is max. w.maxBlockSize big.\n\t// Used for caching leftover data between writes.\n\trbuf *bytes.Buffer\n\n\t// Index of the currently written block.\n\tblockCount uint64\n\n\t// True after the first write.\n\theaderWritten bool\n\n\t// w.maxBlockSize is the maximum number of bytes a single payload may have\n\tmaxBlockSize int64\n\n\t// Used encryption algorithm\n\tflags Flags\n}\n\n// GoodDecBufferSize returns a buffer size that is suitable for decryption.\nfunc (w *Writer) GoodDecBufferSize() int64 {\n\treturn w.maxBlockSize\n}\n\n// GoodEncBufferSize returns a buffer size that is suitable for encryption.\nfunc (w *Writer) GoodEncBufferSize() int64 {\n\treturn w.maxBlockSize + 40\n}\n\nfunc (w *Writer) emitHeaderIfNeeded() error {\n\tif w.headerWritten {\n\t\treturn nil\n\t}\n\n\tw.headerWritten = true\n\theader := GenerateHeader(w.key, w.maxBlockSize, w.flags)\n\t_, err := w.Writer.Write(header)\n\treturn err\n}\n\nfunc (w *Writer) Write(p []byte) (int, error) {\n\tif err := w.emitHeaderIfNeeded(); err != nil {\n\t\treturn 0, err\n\t}\n\n\tfor int64(w.rbuf.Len()) >= w.maxBlockSize {\n\t\tif _, err := w.flushPack(w.rbuf.Next(int(w.maxBlockSize))); err != nil {\n\t\t\treturn 0, err\n\t\t}\n\t}\n\n\t// Remember left-overs for next write:\n\tif _, err := w.rbuf.Write(p); err != nil {\n\t\treturn 0, nil\n\t}\n\n\t// Fake the amount of data we've written:\n\treturn len(p), nil\n}\n\nfunc (w *Writer) flushPack(pack []byte) (int, error) {\n\t// Create a new Nonce for this block:\n\tbinary.LittleEndian.PutUint64(w.nonce, w.blockCount)\n\n\t// Encrypt the text:\n\tw.encBuf = w.aead.Seal(w.encBuf[:0], w.nonce, pack, nil)\n\n\t// Pass it to the underlying writer:\n\tnNonce, err := w.Writer.Write(w.nonce)\n\tif err != nil {\n\t\treturn nNonce, err\n\t}\n\n\tw.blockCount++\n\tnBuf, err := w.Writer.Write(w.encBuf)\n\treturn nNonce + nBuf, err\n}\n\n// Close the Writer and write any left-over blocks\n// This does not close the underlying data stream.\nfunc (w *Writer) Close() error {\n\tif err := w.emitHeaderIfNeeded(); err != nil {\n\t\treturn err\n\t}\n\n\t// Flush last block of data if any:\n\tfor w.rbuf.Len() > 0 {\n\t\tn := int64(w.rbuf.Len())\n\t\tif n > w.maxBlockSize {\n\t\t\tn = w.maxBlockSize\n\t\t}\n\n\t\tif _, err := w.flushPack(w.rbuf.Next(int(n))); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n\n// ReadFrom writes all readable from `r` into `w`.\n//\n// It is intentend as optimized way to copy the whole stream without\n// unneeded copying in between. io.Copy() will use this function automatically.\n//\n// It returns the number of read bytes and any encountered error (no io.EOF)\nfunc (w *Writer) ReadFrom(r io.Reader) (int64, error) {\n\tif err := w.emitHeaderIfNeeded(); err != nil {\n\t\treturn 0, err\n\t}\n\n\tn, nprev := int64(0), -1\n\tbuf := make([]byte, defaultDecBufferSize)\n\n\t// Check if a previous Write() wrote to rbuf.\n\tif w.rbuf.Len() > 0 {\n\t\treturn 0, ErrMixedMethods\n\t}\n\n\tfor {\n\t\tnread, rerr := io.ReadFull(r, buf)\n\t\tif rerr != nil && rerr != io.EOF && rerr != io.ErrUnexpectedEOF {\n\t\t\treturn n, rerr\n\t\t}\n\n\t\tn += int64(nread)\n\n\t\t// Sanity check: check if previous block was properly aligned:\n\t\tif nprev >= 0 && int64(nprev) != w.maxBlockSize && rerr != io.EOF && rerr != io.ErrUnexpectedEOF {\n\t\t\treturn n, ErrBadBlockSize\n\t\t}\n\n\t\tif nread > 0 {\n\t\t\t_, werr := w.flushPack(buf[:nread])\n\t\t\tw.rbuf.Reset()\n\n\t\t\tif werr != nil {\n\t\t\t\treturn n, werr\n\t\t\t}\n\t\t}\n\n\t\tnprev = nread\n\n\t\tif rerr == io.EOF || rerr == io.ErrUnexpectedEOF {\n\t\t\tbreak\n\t\t}\n\t}\n\n\treturn n, nil\n}\n\n// NewWriter calls NewWriterWithFlagsAndBlockSize with a sane default cipher type\n// and a sane default max block size.\nfunc NewWriter(w io.Writer, key []byte, flags Flags) (*Writer, error) {\n\treturn NewWriterWithBlockSize(w, key, flags, defaultMaxBlockSize)\n}\n\n// NewWriterWithBlockSize returns a new Writer which encrypts data with a\n// certain key. If `compressionFlag` is true, the compression\n// flag in the file header will also be true. Otherwise no compression is done.\nfunc NewWriterWithBlockSize(w io.Writer, key []byte, flags Flags, maxBlockSize int64) (*Writer, error) {\n\tew := &Writer{\n\t\tWriter: w,\n\t\trbuf: &bytes.Buffer{},\n\t\tmaxBlockSize: maxBlockSize,\n\t\tflags: flags,\n\t}\n\n\tcipherBit, err := cipherTypeBitFromFlags(flags)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := ew.initAeadCommon(key, cipherBit, ew.maxBlockSize); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn ew, nil\n}\n" }, { "path": "catfs/mio/pagecache/cache.go", "content": "package pagecache\n\nimport (\n\t\"github.com/sahib/brig/catfs/mio/pagecache/page\"\n)\n\n// Cache is the backing layer that stores pages in memory\n// or whatever medium it choses to use.\ntype Cache interface {\n\t// Lookup returns a cached page, identified by `inode` and `page`.\n\t// If there is no such page page.ErrCacheMiss is returned.\n\tLookup(inode int64, page uint32) (*page.Page, error)\n\n\t// Merge the existing cache contents with the new write\n\t// to `pageID`, starting at `pageOff` and with the contents of `buf`.\n\tMerge(inode int64, pageID, pageOff uint32, buf []byte) error\n\n\t// Evict clears cached pages for `inode`. `size` can be used\n\t// to clear only up to a certain size.\n\tEvict(inode, size int64) error\n\n\t// Close the cache and free up all resources.\n\tClose() error\n}\n" }, { "path": "catfs/mio/pagecache/doc.go", "content": "// Package pagecache implements a io.ReaderAt and io.WriterAt that is similar in\n// function to the OverlayFS of Linux. It overlays a read-only stream and\n// enables write support. The writes will take priority on the data in stream\n// and will therefore be visible when calling ReadAt() of the overlay.\n// Read() and Write() are currently not supported, since they would not be used\n// by brig.\n//\n// Note that the normal POSIX file operations are supported. This includes\n// truncating a file to a certain length and also extending it to a certain\n// length. If length of the overlay is greater than the size of the underlying\n// stream we pad it with zeros - just like the kernel would do. Files can be\n// also extended by writing new blocks to the end of the overlay.\n//\n// Seeking will be done when necessary. WriteAt() has to do no seeking at all,\n// while ReadAt() will only seek if it has to (i.e. not reading from cache\n// alone, or if we're not if the right offset already).\n//\n// Implementation detail: The stream is divided into same-sized pages. Each\n// page can be retrieved as whole from the cache. If a page for a certain read\n// offset is found, then ReadAt() will overlay it with the underlying stream or\n// even read from memory if the stream completely occludes the underlying\n// stream. In general, care was taken to optimize a bit more for Write() since\n// pages delivered by ReadAt() can be cached by the FUSE filesystem.\n//\n// You can choose the page cache when creating the overlay. Depending on the\n// page cache implementation it's also possible to edit large files and\n// make edits persistent.\n//\n// NOTE: Whenever uint32 is used in this code, it refers to per-page offsets or\n// size. When int64 is used the content is an offset of the underlying offset.\npackage pagecache\n" }, { "path": "catfs/mio/pagecache/mdcache/l1.go", "content": "package mdcache\n\nimport (\n\t\"container/list\"\n\t\"fmt\"\n\n\t\"github.com/sahib/brig/catfs/mio/pagecache/page\"\n)\n\n// L1 is a pure in-memory LRU cache which does no copying.\n// I did go for LRU because it's insanely simple and easy to implement\n// while still being quite effective.\n//\n// NOTE: We do not use one of the popular caching library here, since\n// none of them seem to fit our use-case. We require the following properties:\n//\n// 1. We must notice when items get evicted (in order to write to l2)\n// 2. We must be able to set a max memory bound.\n// 3. We must avoid copying of pages due to performance reasons.\n//\n// The most popular libraries fail always one of the criteria:\n//\n// - fastcache: fails 1 and 3.\n// - ristretto: fails 1.\n// - bigcache: fails 3.\n//\n// Since we know what kind of data we cache, it is reasonable to implement\n// a very basic LRU cache for L1. Therefore we just use sync.Map here.\n// Oh, and the l1cache is not thread safe, but dircache.go does locking.\n\ntype l1item struct {\n\tPage *page.Page\n\tLink *list.Element\n}\n\ntype l1cache struct {\n\tm map[pageKey]l1item\n\tk *list.List\n\tl2 cacheLayer\n\tmaxMemory int64\n}\n\nfunc newL1Cache(l2 cacheLayer, maxMemory int64) (*l1cache, error) {\n\treturn &l1cache{\n\t\tmaxMemory: maxMemory,\n\t\tl2: l2,\n\t\tk: list.New(),\n\t\tm: make(map[pageKey]l1item),\n\t}, nil\n}\n\nfunc (c *l1cache) Set(pk pageKey, p *page.Page) error {\n\texistingItem, ok := c.m[pk]\n\tif !ok {\n\t\t// new content:\n\t\tc.m[pk] = l1item{\n\t\t\tPage: p,\n\t\t\tLink: c.k.PushBack(pk),\n\t\t}\n\t} else {\n\t\t// do not push another page key,\n\t\t// c.k needs to have unique keys only.\n\t\tc.m[pk] = l1item{\n\t\t\tPage: p,\n\t\t\tLink: existingItem.Link,\n\t\t}\n\n\t\t// prioritize this one more.\n\t\tc.k.MoveToBack(existingItem.Link)\n\t}\n\n\tmaxPages := c.maxMemory / (page.Size + page.Meta)\n\tif int64(len(c.m)) > maxPages {\n\t\toldPkIface := c.k.Remove(c.k.Front())\n\t\toldPk, ok := oldPkIface.(pageKey)\n\t\tif !ok {\n\t\t\treturn fmt.Errorf(\"non-pagekey type stored in l1 keys: %T\", oldPkIface)\n\t\t}\n\n\t\toldItem, ok := c.m[oldPk]\n\t\tdelete(c.m, oldPk)\n\t\tif !ok {\n\t\t\t// c.m and c.k got out of sync.\n\t\t\t// this is very likely a bug.\n\t\t\treturn fmt.Errorf(\"l1: key in key list, but not in map: %v\", oldPk)\n\t\t}\n\n\t\tif c.l2 == nil {\n\t\t\t// nil-interface for l2: loose pages in that case.\n\t\t\t// that may be valid if no disk can be used.\n\t\t\treturn nil\n\t\t}\n\n\t\t// move old page to more persistent cache layer:\n\t\treturn c.l2.Set(oldPk, oldItem.Page)\n\t}\n\n\treturn nil\n}\n\nfunc (c *l1cache) Get(pk pageKey) (*page.Page, error) {\n\titem, ok := c.m[pk]\n\tif !ok {\n\t\treturn nil, page.ErrCacheMiss\n\t}\n\n\t// Sort recently fetched item to end of list:\n\tc.k.MoveToBack(item.Link)\n\treturn item.Page, nil\n}\n\nfunc (c *l1cache) Del(pks []pageKey) {\n\tfor _, pk := range pks {\n\t\tdelItem, ok := c.m[pk]\n\t\tif ok {\n\t\t\tc.k.Remove(delItem.Link)\n\t\t\tdelete(c.m, pk)\n\t\t}\n\t}\n}\n\nfunc (c *l1cache) Close() error {\n\t// help GC if caller somehow still retains a reference:\n\tc.m = nil\n\tc.k = nil\n\treturn nil\n}\n" }, { "path": "catfs/mio/pagecache/mdcache/l1_test.go", "content": "package mdcache\n\nimport (\n\t\"testing\"\n\n\t\"github.com/sahib/brig/catfs/mio/pagecache/page\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc withL1Cache(t *testing.T, fn func(l1, backing *l1cache)) {\n\t// some fake in-mem cache that stores everything that got removed\n\t// out of l1 due to size restrictions.\n\tbacking, err := newL1Cache(nil, int64(^uint64(0)>>1))\n\trequire.NoError(t, err)\n\n\tl1, err := newL1Cache(backing, 4*(page.Size+page.Meta))\n\trequire.NoError(t, err)\n\n\tfn(l1, backing)\n\n\trequire.NoError(t, l1.Close())\n}\n\nfunc TestL1GetSetDel(t *testing.T) {\n\t// NOTE: Only covers the very basic usage.\n\twithL1Cache(t, func(l1, _ *l1cache) {\n\t\tpk := pageKey{1, 0}\n\t\t_, err := l1.Get(pk)\n\t\trequire.Error(t, page.ErrCacheMiss)\n\n\t\tpset := dummyPage(0, 1024)\n\t\trequire.NoError(t, l1.Set(pk, pset))\n\n\t\tpgot, err := l1.Get(pk)\n\t\trequire.NoError(t, err)\n\n\t\trequire.Equal(t, pset.Data, pgot.Data)\n\t\trequire.Equal(t, pset.Extents, pgot.Extents)\n\n\t\tl1.Del([]pageKey{pk})\n\t\t_, err = l1.Get(pk)\n\t\trequire.Error(t, page.ErrCacheMiss)\n\t})\n}\n\nfunc TestL1SwapPriority(t *testing.T) {\n\twithL1Cache(t, func(l1, backing *l1cache) {\n\t\t// Insert 8 pages, only 4 can stay in l1.\n\t\tfor idx := 0; idx < 8; idx++ {\n\t\t\tpk := pageKey{1, uint32(idx)}\n\t\t\trequire.NoError(t, l1.Set(pk, dummyPage(0, uint32((idx+1)*100))))\n\t\t}\n\n\t\tfor idx := 0; idx < 4; idx++ {\n\t\t\tpk := pageKey{1, uint32(idx)}\n\t\t\t_, err := l1.Get(pk)\n\t\t\trequire.Error(t, err, page.ErrCacheMiss)\n\n\t\t\t// should be in backing store, check:\n\t\t\tp, err := backing.Get(pk)\n\t\t\trequire.NoError(t, err)\n\n\t\t\texpected := dummyPage(0, uint32((idx+1)*100))\n\t\t\trequire.Equal(t, expected, p)\n\t\t}\n\n\t\tfor idx := 4; idx < 8; idx++ {\n\t\t\tpk := pageKey{1, uint32(idx)}\n\t\t\tp, err := l1.Get(pk)\n\t\t\trequire.NoError(t, err)\n\n\t\t\texpected := dummyPage(0, uint32((idx+1)*100))\n\t\t\trequire.Equal(t, expected, p)\n\t\t}\n\t})\n}\n" }, { "path": "catfs/mio/pagecache/mdcache/l2.go", "content": "package mdcache\n\nimport (\n\t\"io/ioutil\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"sync\"\n\n\t\"github.com/golang/snappy\"\n\t\"github.com/sahib/brig/catfs/mio/pagecache/page\"\n)\n\n// NOTE: Room for improvement:\n// Introduce heuristic for snappy compression:\n// When an inode does not yield any noticeable compression,\n// disable it for this specific inode. Protects against\n// wasting CPU on already compressed data.\n\ntype l2cache struct {\n\tmu sync.Mutex\n\tdir string\n\tcompress bool\n\tzipBuf []byte\n}\n\n// NOTE: an empty (nil) l2cache is valid, but will not do anything. If an\n// empty string for `dir` is given, such an empty l2cache will be returned.\nfunc newL2Cache(dir string, compress bool) (*l2cache, error) {\n\tif dir == \"\" {\n\t\treturn nil, nil\n\t}\n\n\tvar zipBuf []byte\n\tif compress {\n\t\tzipBuf = make([]byte, snappy.MaxEncodedLen(page.Size))\n\t}\n\n\treturn &l2cache{\n\t\tdir: dir,\n\t\tcompress: compress,\n\t\tzipBuf: zipBuf,\n\t}, nil\n}\n\nfunc (c *l2cache) Set(pk pageKey, p *page.Page) error {\n\tif c == nil {\n\t\treturn nil\n\t}\n\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\n\tdata := p.AsBytes()\n\tif c.compress {\n\t\tdata = snappy.Encode(c.zipBuf, p.AsBytes())\n\t}\n\n\tpath := filepath.Join(c.dir, pk.String())\n\treturn ioutil.WriteFile(path, data, 0600)\n}\n\nfunc (c *l2cache) Get(pk pageKey) (*page.Page, error) {\n\tif c == nil {\n\t\treturn nil, page.ErrCacheMiss\n\t}\n\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\n\tpath := filepath.Join(c.dir, pk.String())\n\tpdata, err := ioutil.ReadFile(path)\n\tif err != nil {\n\t\treturn nil, page.ErrCacheMiss\n\t}\n\n\tif c.compress {\n\t\tpdata, err = snappy.Decode(c.zipBuf, pdata)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn page.FromBytes(pdata)\n}\n\nfunc (c *l2cache) Del(pks []pageKey) {\n\tif c == nil {\n\t\treturn\n\t}\n\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\n\tfor _, pk := range pks {\n\t\tpath := filepath.Join(c.dir, pk.String())\n\n\t\t// no error handling, just get rid of things.\n\t\t// we can't do anything if it could not be deleted.\n\t\tos.Remove(path)\n\t}\n}\n\nfunc (c *l2cache) Close() error {\n\tif c == nil {\n\t\treturn nil\n\t}\n\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\n\treturn os.RemoveAll(c.dir)\n}\n" }, { "path": "catfs/mio/pagecache/mdcache/l2_test.go", "content": "package mdcache\n\nimport (\n\t\"io/ioutil\"\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/sahib/brig/catfs/mio/pagecache/page\"\n\t\"github.com/sahib/brig/util/testutil\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc dummyPage(off, length uint32) *page.Page {\n\tbuf := testutil.CreateDummyBuf(int64(length))\n\treturn page.New(off, buf)\n}\n\nfunc withL2Cache(t *testing.T, fn func(l2 *l2cache)) {\n\tfor _, compress := range []bool{false, true} {\n\n\t\ttmpDir, err := ioutil.TempDir(\"\", \"brig-page-l2\")\n\t\trequire.NoError(t, err)\n\t\tdefer os.RemoveAll(tmpDir)\n\n\t\tl2, err := newL2Cache(tmpDir, compress)\n\t\trequire.NoError(t, err)\n\n\t\ttname := \"no-compress\"\n\t\tif compress {\n\t\t\ttname = \"compress\"\n\t\t}\n\n\t\tt.Run(tname, func(t *testing.T) {\n\t\t\tfn(l2)\n\t\t})\n\n\t\t// double check we do not waste any storage:\n\t\trequire.NoError(t, l2.Close())\n\t\t_, err = os.Stat(tmpDir)\n\t\trequire.True(t, os.IsNotExist(err))\n\t}\n}\n\nfunc TestL2GetSetDel(t *testing.T) {\n\twithL2Cache(t, func(l2 *l2cache) {\n\t\tpk := pageKey{1, 0}\n\t\t_, err := l2.Get(pk)\n\t\trequire.Error(t, page.ErrCacheMiss)\n\n\t\tpset := dummyPage(0, 1024)\n\t\trequire.NoError(t, l2.Set(pk, pset))\n\n\t\tpgot, err := l2.Get(pk)\n\t\trequire.NoError(t, err)\n\n\t\trequire.Equal(t, pset.Data, pgot.Data)\n\t\trequire.Equal(t, pset.Extents, pgot.Extents)\n\n\t\tl2.Del([]pageKey{pk})\n\t\t_, err = l2.Get(pk)\n\t\trequire.Error(t, page.ErrCacheMiss)\n\t})\n}\n\nfunc TestL2Nil(t *testing.T) {\n\t// l2 is optional, so a nil l2 cache should \"work\":\n\tl2, err := newL2Cache(\"\", false)\n\trequire.NoError(t, err)\n\n\t_, err = l2.Get(pageKey{0, 1})\n\trequire.Error(t, page.ErrCacheMiss)\n\n\trequire.NoError(t, l2.Set(pageKey{0, 1}, dummyPage(0, 1024)))\n\tl2.Del([]pageKey{{0, 1}})\n}\n" }, { "path": "catfs/mio/pagecache/mdcache/mdcache.go", "content": "// Package mdcache implements a leveled memory/disk cache combination.\npackage mdcache\n\nimport (\n\t\"fmt\"\n\t\"sync\"\n\n\t\"github.com/sahib/brig/catfs/mio/pagecache/page\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// Options give room for finetuning the behavior of Memory/Disk cache.\ntype Options struct {\n\t// MaxMemoryUsage of L1 in bytes\n\tMaxMemoryUsage int64\n\n\t// SwapDirectory specifies where L2 pages are stored.\n\t// If empty, no l2 cache is used. Instead another l1 cache\n\t// is used in its place, rendering MaxMemoryUsage useless.\n\t// You have to set both for an effect.\n\tSwapDirectory string\n\n\t// L1CacheMissRefill will propagate\n\t// data from L2 to L1 if it could be found\n\t// successfully.\n\tL1CacheMissRefill bool\n\n\t// L2Compress will compress on-disk pages with snappy and decompress them\n\t// on load. Reduces storage, but increases CPU usage if you're swapping.\n\t// Since swapping is slow anyways this is recommended.\n\tL2Compress bool\n}\n\ntype cacheLayer interface {\n\tGet(pk pageKey) (*page.Page, error)\n\tSet(pk pageKey, p *page.Page) error\n\tDel(pks []pageKey)\n\tClose() error\n}\n\n// MDCache is a leveled Memory/Disk cache combination.\ntype MDCache struct {\n\tmu sync.Mutex\n\tl1 cacheLayer\n\tl2 cacheLayer\n\topts Options\n}\n\ntype pageKey struct {\n\tinode int64\n\tpageIdx uint32\n}\n\nfunc (pk pageKey) String() string {\n\treturn fmt.Sprintf(\"%08x-%08x\", pk.inode, pk.pageIdx)\n}\n\n// New returns a new Memory/Disk cache\nfunc New(opts Options) (*MDCache, error) {\n\tl2, err := newL2Cache(opts.SwapDirectory, opts.L2Compress)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tvar l2Iface cacheLayer = l2\n\tif l2 == nil {\n\t\t// special case: when we don't have a l2 cache\n\t\t// then use another memory cache as backing,\n\t\t// with infinite memory.\n\t\tmaxMemory := int64(^uint64(0) >> 1)\n\t\tl2Iface, _ = newL1Cache(nil, maxMemory)\n\t}\n\n\tl1, err := newL1Cache(l2Iface, opts.MaxMemoryUsage)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &MDCache{\n\t\tl1: l1,\n\t\tl2: l2Iface,\n\t\topts: opts,\n\t}, nil\n}\n\n// Lookup implements pagecache.Cache\nfunc (dc *MDCache) Lookup(inode int64, pageIdx uint32) (*page.Page, error) {\n\tdc.mu.Lock()\n\tdefer dc.mu.Unlock()\n\n\treturn dc.get(pageKey{inode: inode, pageIdx: pageIdx})\n}\n\nfunc (dc *MDCache) get(pk pageKey) (*page.Page, error) {\n\tp, err := dc.l1.Get(pk)\n\tswitch err {\n\tcase nil:\n\t\treturn p, nil\n\tcase page.ErrCacheMiss:\n\t\tp, err = dc.l2.Get(pk)\n\t\tif err != nil {\n\t\t\treturn p, err\n\t\t}\n\n\t\tif dc.opts.L1CacheMissRefill {\n\t\t\t// propagate back to l1 cache:\n\t\t\tif err := dc.l1.Set(pk, p); err != nil {\n\t\t\t\treturn p, err\n\t\t\t}\n\t\t}\n\n\t\treturn p, err\n\tdefault:\n\t\treturn nil, err\n\t}\n}\n\n// Merge implements pagecache.Cache\nfunc (dc *MDCache) Merge(inode int64, pageIdx, off uint32, write []byte) error {\n\tdc.mu.Lock()\n\tdefer dc.mu.Unlock()\n\n\tif len(write) == 0 {\n\t\t// empty write deserves no extra computation.\n\t\treturn nil\n\t}\n\n\tif off+uint32(len(write)) > page.Size {\n\t\treturn fmt.Errorf(\"merge: write overflows page bounds\")\n\t}\n\n\tpk := pageKey{inode: inode, pageIdx: pageIdx}\n\tp, err := dc.get(pk)\n\tif err != nil && err != page.ErrCacheMiss {\n\t\treturn err\n\t}\n\n\tif p == nil {\n\t\t// Page was not cached yet.\n\t\t// Create an almost empty page.\n\t\tp = page.New(off, write)\n\t}\n\n\tp.Overlay(off, write)\n\treturn dc.l1.Set(pk, p)\n}\n\n// Evict implements pagecache.Cache\nfunc (dc *MDCache) Evict(inode, size int64) error {\n\tdc.mu.Lock()\n\tdefer dc.mu.Unlock()\n\n\t// Figure out all possible indices from size:\n\tpks := []pageKey{}\n\tpageHi := uint32(size / page.Size)\n\tif size%page.Size > 0 {\n\t\tpageHi++\n\t}\n\n\tfor pageIdx := uint32(0); pageIdx < pageHi; pageIdx++ {\n\t\tpks = append(pks, pageKey{inode: inode, pageIdx: pageIdx})\n\t}\n\n\tdc.l1.Del(pks)\n\tdc.l2.Del(pks)\n\treturn nil\n}\n\n// Close closes the cache contents and cleans up resources.\nfunc (dc *MDCache) Close() error {\n\tdc.mu.Lock()\n\tdefer dc.mu.Unlock()\n\n\tif err := dc.l1.Close(); err != nil {\n\t\tlog.WithError(err).Warnf(\"failed to reset l1 cache\")\n\t}\n\n\treturn dc.l2.Close()\n}\n" }, { "path": "catfs/mio/pagecache/mdcache/mdcache_test.go", "content": "package mdcache\n\nimport (\n\t\"io/ioutil\"\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/sahib/brig/catfs/mio/pagecache/page\"\n\t\"github.com/sahib/brig/util/testutil\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc withMDCache(t *testing.T, fn func(mdc *MDCache)) {\n\ttmpDir, err := ioutil.TempDir(\"\", \"brig-page-l2\")\n\trequire.NoError(t, err)\n\tdefer os.RemoveAll(tmpDir)\n\n\tmd, err := New(Options{\n\t\tMaxMemoryUsage: 4 * page.Size,\n\t\tSwapDirectory: tmpDir,\n\t\tL1CacheMissRefill: true,\n\t})\n\n\trequire.NoError(t, err)\n\n\tfn(md)\n\n\trequire.NoError(t, md.Close())\n}\n\nfunc TestMDBasic(t *testing.T) {\n\twithMDCache(t, func(mdc *MDCache) {\n\t\tfor idx := 0; idx < 8; idx++ {\n\t\t\terr := mdc.Merge(1, uint32(idx), 0, testutil.CreateDummyBuf(page.Size))\n\t\t\trequire.NoError(t, err)\n\t\t}\n\n\t\tfor idx := 0; idx < 8; idx++ {\n\t\t\tp, err := mdc.Lookup(1, uint32(idx))\n\t\t\trequire.NoError(t, err)\n\n\t\t\trequire.Equal(t, testutil.CreateDummyBuf(page.Size), p.Data)\n\t\t\trequire.Equal(t, []page.Extent{{\n\t\t\t\tOffLo: 0,\n\t\t\t\tOffHi: page.Size,\n\t\t\t}}, p.Extents)\n\t\t}\n\n\t\trequire.NoError(t, mdc.Evict(1, 8*page.Size))\n\t})\n}\n" }, { "path": "catfs/mio/pagecache/overlay.go", "content": "package pagecache\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"sync\"\n\n\t\"github.com/sahib/brig/catfs/mio/pagecache/page\"\n\t\"github.com/sahib/brig/util\"\n)\n\n// Layer is a layer above a read-only stream with write support.\ntype Layer struct {\n\t// underlying stream\n\trs io.ReadSeeker\n\n\t// inode is a unique identifier for the stream.\n\t// it is used as identifier in the page cache.\n\tinode int64\n\n\t// cache gives access to cached pages\n\tcache Cache\n\n\t// size is the number of bytes that can be read from\n\t// `rs` from start to end. It represents the \"old\" file size.\n\t// It's only use to decide when to stop reading from the\n\t// underlying stream. For deciding where EOF is, length is used.\n\tsize int64\n\n\t// length starts out same as size, but might change due to\n\t// calls to Truncate(). Truncate is terrible name since it\n\t// can be also used to extend a file's length. But that's\n\t// how the underlying syscall is named, so we follow that.\n\tlength int64\n\n\t// overlayOffset is the last known offset in the stream,\n\t// including reads from the cache. It is the position in the\n\t// overlayed stream.\n\toverlayOffset int64\n\n\t// streamOffset indicates the offset in the underlying stream `rs`.\n\t// It can be the same as `overlayOffset` but is not most of the time.\n\t// Not counted in in `streamOffset` are bytes that were read from\n\t// the cache exclusively, with no need to read from `rs`.\n\t// It's not updated when data is purely read from the cache.\n\tstreamOffset int64\n}\n\n// NewLayer returns a paged overlay for `rs`, reading and storing data from\n// `cache`. `inode` will be used as cache identifier for this file. The only\n// need is that it is unique to this file, otherwise it does not need any\n// inode-like semantics. `size` must be known in advance and reflects the size\n// of `rs`. This cannot be used for pure streaming. `rs` is assumed to be positioned\n// at the zero offset. If not, subtract the offset from `size`.\nfunc NewLayer(rs io.ReadSeeker, cache Cache, inode, size int64) (*Layer, error) {\n\tif err := cache.Evict(inode, size); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &Layer{\n\t\trs: rs,\n\t\tinode: inode,\n\t\tsize: size,\n\t\tlength: size,\n\t\tcache: cache,\n\t}, nil\n}\n\nfunc (l *Layer) ensureOffset(zpr *zeroPadReader) error {\n\tif l.overlayOffset == l.streamOffset {\n\t\treturn nil\n\t}\n\n\tzpr.off = l.overlayOffset\n\tnewOffset, err := l.rs.Seek(l.overlayOffset, io.SeekStart)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tl.streamOffset = newOffset\n\tif newOffset != l.overlayOffset {\n\t\treturn fmt.Errorf(\"page: ensure offset failed (want: %d, got %d)\", l.overlayOffset, newOffset)\n\t}\n\n\treturn nil\n}\n\n// WriteAt writes `buf` to `off`. It will appear on the next\n// read operation.\nfunc (l *Layer) WriteAt(buf []byte, off int64) (n int, err error) {\n\t// If `buf` is large enough to span over several writes then we\n\t// have to calculate the offset of the first page, so that new\n\t// data is written to the correct place.\n\tpageOff := off % page.Size\n\tpageBuf := buf\n\n\t// Go over all pages this write affects.\n\tnewOff := off + int64(len(buf))\n\tpageLo := off / page.Size\n\tpageHi := newOff / page.Size\n\tif newOff%page.Size == 0 {\n\t\tpageHi--\n\t}\n\n\tfor pageIdx := pageLo; pageIdx <= pageHi; pageIdx++ {\n\t\t// Divide `buf` into small portions that will be copied\n\t\t// to the individual pages.\n\n\t\tmayWrite := page.Size - pageOff\n\t\tif mayWrite > int64(len(pageBuf)) {\n\t\t\tmayWrite = int64(len(pageBuf))\n\t\t}\n\n\t\tif mayWrite == 0 {\n\t\t\tbreak\n\t\t}\n\n\t\t// Overlay the part of `buf` that affects this page\n\t\t// and merge with any pre-existing writes.\n\t\tif err := l.cache.Merge(\n\t\t\tl.inode,\n\t\t\tuint32(pageIdx),\n\t\t\tuint32(pageOff),\n\t\t\tpageBuf[:mayWrite],\n\t\t); err != nil {\n\t\t\treturn -1, err\n\t\t}\n\n\t\t// starting from the second block the page offset will\n\t\t// be always zero. That's only relevant for len(buf) > page.Size.\n\t\tpageOff = 0\n\t\tpageBuf = pageBuf[mayWrite:]\n\t}\n\n\t// check if this write extended the full buffer.\n\t// If so we need to remember the new length.\n\tif newOff := off + int64(len(buf)); newOff > l.length {\n\t\tl.length = newOff\n\t}\n\n\t// We always write the full buffer or fail in prior.\n\treturn len(buf), nil\n}\n\nvar (\n\tcopyBufPool = &sync.Pool{\n\t\tNew: func() interface{} {\n\t\t\treturn make([]byte, page.Size)\n\t\t},\n\t}\n)\n\n// ReadAt reads into `buf` from the position `off`.\n//\n// NOTE: There are two things that are not implemented\n// according to the io.ReaderAt docs:\n//\n// * ReadAt() should not modify the seek offset.\n// This implementation however does this.\n// * ReadAt() must be allowed to call in parallel.\n// We cannot guarantee that at the moment since sometimes\n// we have to seek the underlying stream - mutex?\nfunc (l *Layer) ReadAt(buf []byte, off int64) (int, error) {\n\t// when requesting reads beyond the size of the overlay,\n\t// we should immediately cancel the request.\n\tif off >= l.length {\n\t\treturn 0, io.EOF\n\t}\n\n\t// set the desired offset\n\tl.overlayOffset = off\n\n\t// small helper for copying data to buf.\n\t// we will never copy more than page.Size to buf.\n\tib := &iobuf{dst: buf}\n\n\t// l.rs might not be as long as l.length.\n\t// We need to pad the rest of the stream with zeros.\n\t// This reader does this.\n\tzpr := &zeroPadReader{\n\t\tr: l.rs,\n\t\toff: off,\n\t\tsize: l.size,\n\t\tlength: l.length,\n\t}\n\n\tpageOff := uint32(off % page.Size)\n\n\t// keep the copy buf around between GC runs.\n\tcopyBuf := copyBufPool.Get().([]byte)\n\tdefer copyBufPool.Put(copyBuf)\n\n\t// Go over all pages this read may affect.\n\t// We might return early due to io.EOF though.\n\tnewOff := off + int64(len(buf))\n\tpageLo := off / page.Size\n\tpageHi := newOff / page.Size\n\tif newOff%page.Size == 0 {\n\t\tpageHi--\n\t}\n\n\tfor pageIdx := pageLo; pageIdx <= pageHi && ib.Left() > 0; pageIdx++ {\n\t\tpageMax := uint32(util.Min64(l.length, l.overlayOffset+page.Size) - l.overlayOffset)\n\t\tif pageMax+pageOff > page.Size {\n\t\t\tpageMax = page.Size - pageOff\n\t\t}\n\n\t\tp, err := l.cache.Lookup(l.inode, uint32(pageIdx))\n\t\tswitch err {\n\t\tcase page.ErrCacheMiss:\n\t\t\t// we don't have this page cached.\n\t\t\t// need to read it from zpr directly.\n\t\t\tif err := l.ensureOffset(zpr); err != nil {\n\t\t\t\treturn ib.Len(), err\n\t\t\t}\n\n\t\t\tn, err := copyNBuffer(ib, zpr, util.Min64(int64(ib.Left()), int64(pageMax)), copyBuf)\n\t\t\tl.overlayOffset += n\n\t\t\tl.streamOffset += n\n\n\t\t\tif err != nil {\n\t\t\t\treturn ib.Len(), err\n\t\t\t}\n\n\t\t\t// NOTE: we could be clever here and cache pages that have\n\t\t\t// been read often. We could even hook in things like\n\t\t\t// fadvise() into this layer.\n\t\tcase nil:\n\t\t\t// In this case we know that the page is cached.\n\t\t\t// We can fill `buf` with the page of the data,\n\t\t\t// (provided by page.Reader()).\n\t\t\toccludesStream := p.OccludesStream(pageOff, pageMax)\n\t\t\tif !occludesStream {\n\t\t\t\t// only seek if we have to.\n\t\t\t\tif err := l.ensureOffset(zpr); err != nil {\n\t\t\t\t\treturn ib.Len(), err\n\t\t\t\t}\n\n\t\t\t\tpageN, err := io.ReadFull(zpr, copyBuf[pageOff:])\n\n\t\t\t\t// Still handle the data even in case of errors.\n\t\t\t\tp.Underlay(pageOff, copyBuf[pageOff:pageOff+uint32(pageN)])\n\t\t\t\tl.streamOffset += int64(pageN)\n\n\t\t\t\tif err != nil && err != io.EOF && err != io.ErrUnexpectedEOF {\n\t\t\t\t\treturn ib.Len(), err\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tr := bytes.NewReader(p.Data[pageOff : pageOff+pageMax])\n\t\t\tn, err := copyNBuffer(ib, r, int64(ib.Left()), copyBuf)\n\t\t\tif err != nil && err != io.EOF {\n\t\t\t\treturn ib.Len(), err\n\t\t\t}\n\n\t\t\tl.overlayOffset += n\n\t\tdefault:\n\t\t\t// some other error during cache lookup.\n\t\t\treturn ib.Len(), err\n\t\t}\n\n\t\t// If read spans over several pages, the second\n\t\t// page has to start at zero.\n\t\tpageOff = 0\n\t}\n\n\treturn ib.Len(), nil\n}\n\n// Truncate sets the size of the stream.\n// There are three cases:\n//\n// - `size` is equal to Length(): Nothing happens.\n// - `size` is less than Length(): The stream will return io.EOF earlier.\n// - `size` is more than Length(): The stream will be padded with zeros.\n//\n// This matches the behavior of the equally confusingly named POSIX\n// ftruncate() function. Note that Truncate() is a very fast operation.\nfunc (l *Layer) Truncate(size int64) {\n\tl.length = size\n}\n\n// Length is the current truncated length of the overlay.\n// When you did not call Truncate() it will be the size you\n// passed to NewLayer(). Otherwise it is what you passed\n// to the last call of Truncate().\nfunc (l *Layer) Length() int64 {\n\treturn l.length\n}\n\n/////////////////////////////////////\n// FILE I/O COMPATIBILITY METHODS //\n// //\n// Do not use, unless you have to. //\n// Prefer WriteAt() & ReadAt() //\n/////////////////////////////////////\n\n// Read implements io.Reader by calling ReadAt()\n// with the current offset.\nfunc (l *Layer) Read(buf []byte) (int, error) {\n\treturn l.ReadAt(buf, l.overlayOffset)\n}\n\n// Write writes `buf` at the current offset.\n// It does NOT modify the read position.\nfunc (l *Layer) Write(buf []byte) (int, error) {\n\tn, err := l.WriteAt(buf, l.overlayOffset)\n\treturn n, err\n}\n\n// Seek changes the current offset for Write and Read.\n// Note that in this implementation calling ReadAt.\n// *does* change the seek offset. Use Seek() to make\n// sure you're reading from the right spot.\nfunc (l *Layer) Seek(off int64, whence int) (int64, error) {\n\tswitch whence {\n\tcase io.SeekStart:\n\t\tl.overlayOffset = off\n\tcase io.SeekCurrent:\n\t\tl.overlayOffset += off\n\tcase io.SeekEnd:\n\t\tl.overlayOffset = l.length + off\n\tdefault:\n\t\treturn 0, fmt.Errorf(\"invalid whence %d\", whence)\n\t}\n\n\treturn l.overlayOffset, nil\n}\n\n// Close will close the overlay and free up all resources,\n// including pages in the cache.\nfunc (l *Layer) Close() error {\n\treturn l.cache.Evict(l.inode, l.length)\n}\n\n// WriteTo implements io.WriterTo\nfunc (l *Layer) WriteTo(w io.Writer) (int64, error) {\n\t// NOTE: This method is mostly used in tests.\n\t// but can be also used by io.Copy() internally.\n\t// There is room for optimizations here:\n\t// Avoid one copy by directly writing to copyBuf.\n\tcopyBuf := copyBufPool.Get().([]byte)\n\tdefer copyBufPool.Put(copyBuf)\n\n\twsum := int64(0)\n\n\tfor {\n\t\trn, rerr := l.ReadAt(copyBuf, l.overlayOffset)\n\t\tif rerr != nil && rerr != io.EOF {\n\t\t\treturn wsum, rerr\n\t\t}\n\n\t\twn, werr := w.Write(copyBuf[:rn])\n\t\twsum += int64(wn)\n\t\tif werr != nil {\n\t\t\treturn wsum, werr\n\t\t}\n\n\t\tif wn < rn {\n\t\t\treturn wsum, io.ErrShortWrite\n\t\t}\n\n\t\tif rerr == io.EOF {\n\t\t\treturn wsum, nil\n\t\t}\n\n\t\tif rn == 0 {\n\t\t\treturn wsum, fmt.Errorf(\"nothing read, but no EOF\")\n\t\t}\n\t}\n\n}\n" }, { "path": "catfs/mio/pagecache/overlay_test.go", "content": "package pagecache\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"math/rand\"\n\t\"testing\"\n\n\t\"github.com/sahib/brig/catfs/mio/pagecache/mdcache\"\n\t\"github.com/sahib/brig/catfs/mio/pagecache/page\"\n\t\"github.com/sahib/brig/util\"\n\t\"github.com/sahib/brig/util/testutil\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc withLayer(t *testing.T, size int64, fn func(expected []byte, p *Layer)) {\n\tmd, err := mdcache.New(mdcache.Options{\n\t\tMaxMemoryUsage: 4 * page.Size,\n\t\tSwapDirectory: \"\",\n\t})\n\n\trequire.NoError(t, err)\n\n\tdata := testutil.CreateDummyBuf(size)\n\tp, err := NewLayer(bytes.NewReader(data), md, 42, size)\n\trequire.NoError(t, err)\n\n\texpected := make([]byte, size)\n\tcopy(expected, data)\n\tfn(expected, p)\n\n\trequire.NoError(t, md.Close())\n}\n\nvar (\n\ttestSizes = []int64{\n\t\t16*page.Size + 0,\n\t\t16*page.Size - 1,\n\t\t16*page.Size + 1,\n\t\tpage.Size + 0,\n\t\tpage.Size - 1,\n\t\tpage.Size + 1,\n\t\t0,\n\t\t1,\n\t}\n)\n\nfunc TestReadOnly(t *testing.T) {\n\tfor _, testSize := range testSizes {\n\t\tt.Run(fmt.Sprintf(\"%d\", testSize), func(t *testing.T) {\n\t\t\twithLayer(t, testSize, func(expected []byte, p *Layer) {\n\t\t\t\tgot := bytes.NewBuffer([]byte{})\n\t\t\t\tn, err := p.WriteTo(got)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.Equal(t, testSize, n)\n\t\t\t\trequire.Equal(t, expected, got.Bytes())\n\t\t\t})\n\t\t})\n\t}\n}\n\nfunc padOrCutToLength(buf []byte, length int64) []byte {\n\tif int64(len(buf)) >= length {\n\t\treturn buf[:length]\n\t}\n\n\tc := make([]byte, length)\n\tcopy(c, buf)\n\treturn c\n}\n\nfunc TestReadOnlyTruncate(t *testing.T) {\n\ttruncOffsets := []int64{\n\t\t-2*page.Size + 0,\n\t\t-2*page.Size - 1,\n\t\t-2*page.Size + 1,\n\t\t+2*page.Size + 0,\n\t\t+2*page.Size - 1,\n\t\t+2*page.Size + 1,\n\t\t+page.Size + 0,\n\t\t+page.Size - 1,\n\t\t+page.Size + 1,\n\t\t-page.Size + 0,\n\t\t-page.Size - 1,\n\t\t-page.Size + 1,\n\t\t+0,\n\t\t+1,\n\t\t-1,\n\t}\n\n\tfor _, testSize := range testSizes {\n\t\tt.Run(fmt.Sprintf(\"%d\", testSize), func(t *testing.T) {\n\t\t\tfor _, truncOff := range truncOffsets {\n\t\t\t\tlength := util.Max64(0, testSize+truncOff)\n\t\t\t\tif length == testSize {\n\t\t\t\t\t// no need to run test with no truncation.\n\t\t\t\t\t// already covered by TestReadOnly()\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\n\t\t\t\tt.Run(fmt.Sprintf(\"trunc-to-%d\", length), func(t *testing.T) {\n\t\t\t\t\twithLayer(t, testSize, func(expected []byte, p *Layer) {\n\t\t\t\t\t\tgot := bytes.NewBuffer([]byte{})\n\t\t\t\t\t\tp.Truncate(length)\n\n\t\t\t\t\t\tn, err := p.WriteTo(got)\n\t\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\t\trequire.Equal(t, length, n)\n\n\t\t\t\t\t\tres := padOrCutToLength(got.Bytes(), length)\n\t\t\t\t\t\trequire.Equal(\n\t\t\t\t\t\t\tt,\n\t\t\t\t\t\t\tpadOrCutToLength(expected, length),\n\t\t\t\t\t\t\tres,\n\t\t\t\t\t\t)\n\t\t\t\t\t})\n\t\t\t\t})\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestWriteSingle(t *testing.T) {\n\tfor _, testReadSize := range testSizes {\n\t\tt.Run(fmt.Sprintf(\"read-%d\", testReadSize), func(t *testing.T) {\n\t\t\tfor _, testWriteSize := range testSizes {\n\t\t\t\tt.Run(fmt.Sprintf(\"write-%d\", testWriteSize), func(t *testing.T) {\n\t\t\t\t\twithLayer(t, testReadSize, func(expected []byte, p *Layer) {\n\t\t\t\t\t\texpected = testutil.CreateRandomDummyBuf(testWriteSize, 23)\n\t\t\t\t\t\twn, err := p.WriteAt(expected, 0)\n\t\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\t\trequire.Equal(t, int64(wn), testWriteSize)\n\n\t\t\t\t\t\tgot := make([]byte, testWriteSize)\n\t\t\t\t\t\trn, err := p.Read(got)\n\t\t\t\t\t\tif testReadSize == 0 {\n\t\t\t\t\t\t\t// special case: that will immediately return EOF.\n\t\t\t\t\t\t\trequire.Error(t, io.EOF, err)\n\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\t\trequire.Equal(t, wn, rn)\n\t\t\t\t\t})\n\t\t\t\t})\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestWriteRandomOffset(t *testing.T) {\n\t// Randomly generate writes and write them to the layer.\n\t// The randomness is controlled by seed to be reproducible.\n\t// The generated data is also copy()'d to a slice which\n\t// serves as way to check the overlay on the final read.\n\n\tfor seed := 0; seed < 40; seed++ {\n\t\tt.Run(fmt.Sprintf(\"seed-%d\", seed), func(t *testing.T) {\n\t\t\tfor _, testReadSize := range testSizes {\n\t\t\t\tif testReadSize == 0 {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\n\t\t\t\tt.Run(fmt.Sprintf(\"size-%d\", testReadSize*2), func(t *testing.T) {\n\t\t\t\t\twithLayer(t, testReadSize, func(expected []byte, p *Layer) {\n\n\t\t\t\t\t\t// NOTE: We do not write beyond p.Length()\n\t\t\t\t\t\t// to make this test easier to check.\n\t\t\t\t\t\tp.Truncate(testReadSize * 2)\n\n\t\t\t\t\t\texpected = padOrCutToLength(expected, p.Length())\n\t\t\t\t\t\trequire.Equal(t, testReadSize*2, p.Length())\n\n\t\t\t\t\t\trand.Seed(int64(seed))\n\t\t\t\t\t\tfor nwrites := 0; nwrites < seed; nwrites++ {\n\t\t\t\t\t\t\twriteOff := rand.Int63n(p.Length())\n\t\t\t\t\t\t\twriteLen := rand.Int63n(p.Length() - writeOff + 1)\n\n\t\t\t\t\t\t\t// stream contains 0-254 data, overwrite with random:\n\t\t\t\t\t\t\tbuf := testutil.CreateRandomDummyBuf(writeLen, int64(seed))\n\t\t\t\t\t\t\tcopy(expected[writeOff:writeOff+writeLen], buf)\n\t\t\t\t\t\t\twn, err := p.WriteAt(buf, writeOff)\n\t\t\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\t\t\trequire.Equal(t, int(writeLen), wn)\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\tgot := &bytes.Buffer{}\n\t\t\t\t\t\trn, err := io.Copy(got, p)\n\t\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\t\trequire.Equal(t, p.Length(), int64(rn))\n\t\t\t\t\t\trequire.Equal(t, p.Length(), int64(len(expected)))\n\t\t\t\t\t\trequire.Equal(t, p.Length(), int64(got.Len()))\n\n\t\t\t\t\t\t// This for loop here is just for easier digest\n\t\t\t\t\t\t// debug output. require.Equal() outputs huge\n\t\t\t\t\t\t// diffs that are seldom helpful.\n\t\t\t\t\t\tfor idx := 0; idx < got.Len(); idx++ {\n\t\t\t\t\t\t\tif expected[idx] != got.Bytes()[idx] {\n\t\t\t\t\t\t\t\trequire.Equal(\n\t\t\t\t\t\t\t\t\tt,\n\t\t\t\t\t\t\t\t\texpected[idx:idx+256],\n\t\t\t\t\t\t\t\t\tgot.Bytes()[idx:idx+256],\n\t\t\t\t\t\t\t\t)\n\t\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\t// This assert is just here in case the for loop\n\t\t\t\t\t\t// above has a bug or gets lost somehow.\n\t\t\t\t\t\trequire.Equal(t, expected, got.Bytes())\n\t\t\t\t\t})\n\t\t\t\t})\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestReadRandomOffset(t *testing.T) {\n\ttcs := []struct {\n\t\tsize int64\n\t\tlength int64\n\t\tnops int\n\t}{\n\t\t{page.Size, page.Size, 10},\n\t\t{2 * page.Size, page.Size, 20},\n\t\t{page.Size, 2 * page.Size, 30},\n\n\t\t{16 * page.Size, 16 * page.Size, 30},\n\t\t{16 * page.Size, 16*page.Size + 1, 20},\n\t\t{16*page.Size + 1, 16 * page.Size, 10},\n\t}\n\t//\t\t16*page.Size + 0,\n\t//\t\t16*page.Size - 1,\n\t//\t\t16*page.Size + 1,\n\t//\t\tpage.Size + 0,\n\t//\t\tpage.Size - 1,\n\t//\t\tpage.Size + 1,\n\t//\t\t0,\n\t//\t\t1,\n\n\tfor _, tc := range tcs {\n\t\tname := fmt.Sprintf(\"s%d-l%d-o%d\", tc.size, tc.length, tc.nops)\n\t\tt.Run(name, func(t *testing.T) {\n\t\t\t// always use the same writing distribution,\n\t\t\t// just increasingly more writes:\n\n\t\t\twithLayer(t, tc.size, func(expected []byte, p *Layer) {\n\t\t\t\tp.Truncate(tc.length)\n\t\t\t\texpected = padOrCutToLength(expected, p.Length())\n\t\t\t\trequire.Equal(t, tc.length, p.Length())\n\t\t\t\trand.Seed(42)\n\n\t\t\t\toverlayed := make([]byte, len(expected))\n\t\t\t\tcopy(overlayed, expected)\n\n\t\t\t\tfor idx := 0; idx < tc.nops; idx++ {\n\t\t\t\t\twriteOff := rand.Int63n(p.Length())\n\t\t\t\t\twriteLen := rand.Int63n(p.Length() - writeOff + 1)\n\n\t\t\t\t\t// stream contains 0-254 data, overwrite with random:\n\t\t\t\t\tbuf := testutil.CreateRandomDummyBuf(writeLen, int64(42))\n\t\t\t\t\tcopy(overlayed[writeOff:writeOff+writeLen], buf)\n\t\t\t\t\twn, err := p.WriteAt(buf, writeOff)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\trequire.Equal(t, int(writeLen), wn)\n\t\t\t\t}\n\n\t\t\t\tfor seed := 0; seed < 40; seed++ {\n\t\t\t\t\trand.Seed(int64(seed))\n\t\t\t\t\tt.Run(fmt.Sprintf(\"seed-%d\", seed), func(t *testing.T) {\n\t\t\t\t\t\tfor idx := 0; idx < tc.nops; idx++ {\n\t\t\t\t\t\t\treadOff := rand.Int63n(p.Length())\n\t\t\t\t\t\t\treadLen := rand.Int63n(p.Length() - readOff + 1)\n\t\t\t\t\t\t\treadBuf := make([]byte, readLen)\n\n\t\t\t\t\t\t\trn, err := p.ReadAt(readBuf, readOff)\n\t\t\t\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\t\t\t\trequire.Equal(t, readLen, int64(rn))\n\t\t\t\t\t\t\tfor idx := int64(0); idx < readLen; idx++ {\n\t\t\t\t\t\t\t\tif overlayed[readOff+idx] != readBuf[idx] {\n\t\t\t\t\t\t\t\t\trequire.Failf(t, \"bad data read\", \"first wrong offset: %d\", readOff+idx)\n\t\t\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t}\n\n\t\t\t\t\t\t\trequire.Equal(t, overlayed[readOff:readOff+readLen], readBuf)\n\t\t\t\t\t\t}\n\t\t\t\t\t})\n\t\t\t\t}\n\t\t\t})\n\t\t})\n\t}\n}\n" }, { "path": "catfs/mio/pagecache/page/page.go", "content": "package page\n\n// NOTE: I had quite often brain freeze while figuring out the indexing.\n// If you do too, take a piece of paper and draw it.\n// If you don't, congratulations. You're smarter than me.\n\nimport (\n\t\"bytes\"\n\t\"encoding/binary\"\n\t\"errors\"\n\t\"fmt\"\n\t\"sort\"\n\n\tlog \"github.com/sirupsen/logrus\"\n)\n\nconst (\n\t// Size is the default size for a page.\n\t// Last page might be smaller.\n\tSize = 64 * 1024\n\n\t// Meta is the number of bytes we use\n\t// to store the extents of the page.\n\t// (4k is the typical page size on linux)\n\tMeta = 4 * 1024\n\n\t// ExtentSize needed to store a single extent.\n\tExtentSize = 8\n)\n\nvar (\n\t// ErrCacheMiss indicates that a page is missing from the cache.\n\t// Not a real error, but a sentinel to indicate this state.\n\tErrCacheMiss = errors.New(\"cache miss\")\n)\n\n// Extent marks a single write or\n// several writes that were joined to one.\n//\n// The written data is in the range [lo, hi)\n// where hi is not part of the write!\n//\n// In other words, when writing 16384 bytes\n// at OffLo=0, then OffHi=16384, but the last\n// valid bytes is at p.Data[OffHi-1]!\n//\n// This was chosen so you could say p.Data[OffLo:OffHi]\n// and it would do what you would guess it would do.\ntype Extent struct {\n\tOffLo, OffHi uint32\n}\n\nfunc (e Extent) String() string {\n\treturn fmt.Sprintf(\"[%d-%d)\", e.OffLo, e.OffHi)\n}\n\n// Page is a single cached page\ntype Page struct {\n\t// Extents is a list describing where\n\t// `Data` contains valid data.\n\tExtents []Extent\n\n\t// Data is the data hold by the page.\n\t// It is allocated to Size+Meta bytes,\n\t// even when no data was used.\n\tData []byte\n}\n\nfunc (p *Page) String() string {\n\tbuf := &bytes.Buffer{}\n\tfor idx, extent := range p.Extents {\n\t\tbuf.WriteString(extent.String())\n\t\tif idx+1 != len(p.Extents) {\n\t\t\tbuf.WriteString(\", \")\n\t\t}\n\t}\n\n\treturn fmt.Sprintf(\"\", p.Data, buf.String())\n}\n\n// New allocates a new page with an initial extent at `off`\n// and with `write` as data. See also Overlay()\nfunc New(off uint32, write []byte) *Page {\n\t// NOTE: We allocate more than we actually need in order to implement\n\t// AsBytes and FromBytes efficiently without further allocations.\n\tbacking := make([]byte, Size+Meta)\n\tp := &Page{Data: backing[:Size]}\n\tp.Overlay(off, write)\n\treturn p\n}\n\n// FromBytes reconstructs a page from the give data.\n// Note that ownership over the data is taken, do not write\n// to it anymore while using it as a page.\nfunc FromBytes(data []byte) (*Page, error) {\n\tif len(data) < Size {\n\t\treturn nil, fmt.Errorf(\"page data smaller than mandatory size\")\n\t}\n\n\tp := Page{Data: data[:Size]}\n\textents := data[Size:cap(data)]\n\tfor idx := 0; idx < len(extents); idx += ExtentSize {\n\t\tif idx+ExtentSize > len(extents) {\n\t\t\t// sanity check: do not read after extents.\n\t\t\tcontinue\n\t\t}\n\n\t\toffLo := binary.LittleEndian.Uint32(extents[idx+0:])\n\t\toffHi := binary.LittleEndian.Uint32(extents[idx+4:])\n\t\tif offLo == 0 && offHi == 0 {\n\t\t\t// empty writes are invalid and serve as sentinel value\n\t\t\t// to tell us we read too far. No other extents to expect.\n\t\t\tbreak\n\t\t}\n\n\t\tif offLo == offHi {\n\t\t\tlog.Warnf(\"page cache: loaded empty extent\")\n\t\t\tcontinue\n\t\t}\n\n\t\tif offLo > offHi {\n\t\t\tlog.Warnf(\"page cache: loaded invalid extent\")\n\t\t\tcontinue\n\t\t}\n\n\t\tp.Extents = append(p.Extents, Extent{\n\t\t\tOffLo: offLo,\n\t\t\tOffHi: offHi,\n\t\t})\n\t}\n\n\treturn &p, nil\n}\n\n// AsBytes encodes the extents at the end of the page data\n// and returns the full sized page array.\nfunc (p *Page) AsBytes() []byte {\n\tif cap(p.Data) < Size+Meta {\n\t\t// this is a programming error:\n\t\tpanic(fmt.Sprintf(\"bug: page memory was allocated too small %d\", cap(p.Data)))\n\t}\n\n\tpdata := p.Data[:Size+Meta]\n\tpmeta := pdata[Size:]\n\n\tfor idx, extent := range p.Extents {\n\t\toff := idx * ExtentSize\n\t\tif off+ExtentSize >= cap(p.Data)-Size {\n\t\t\t// NOTE: This is an inefficient allocation/copy. It will occur only\n\t\t\t// when there are more than $(Meta/ExtentSize) distinct writes\n\t\t\t// without a single read of this page (a non-occluding read will\n\t\t\t// unify all extents). This is pretty unlikely to happen in normal\n\t\t\t// circumstances. If that happens it's a weird use case, so\n\t\t\t// allocate another 64 extents.\n\t\t\tpdata = append(pdata, make([]byte, ExtentSize*64)...)\n\t\t\tp.Data = pdata[:Size]\n\t\t\tpmeta = pdata[Size:cap(pdata)]\n\t\t}\n\n\t\tbinary.LittleEndian.PutUint32(pmeta[off+0:], extent.OffLo)\n\t\tbinary.LittleEndian.PutUint32(pmeta[off+4:], extent.OffHi)\n\t}\n\n\treturn pdata\n}\n\n// affectedExtentIdxs() returns the indices of extents\n// that would be affected when writing a new extent with\n// the offsets [lo, hi].\n//\n// Consider the following cases, where \"-\" are areas\n// with existing extents, \"_\" without and \"|\" denotes\n// the area where we want to write newly. First extent\n// is called E1, second E2 and so on.\n//\n// Case 1: => min=E2, max=E2 (does not hit any extent)\n//\n// ------__|--|___-------\n//\n// Case 2: => min=E2, max=E3 (partially hits an extent)\n//\n// ------__|-------|-----\n//\n// Case 3: => min=E2, max=E3 (fully inside one extent)\n//\n// ------________--|---|-\n//\n// Case 4: => min=len(extents), max=len(extents) (outside any extent)\n//\n// ------________-------- |-----|\nfunc (p *Page) affectedExtentIdxs(lo, hi uint32) (int, int) {\n\tminExIdx := sort.Search(len(p.Extents), func(i int) bool {\n\t\treturn lo < p.Extents[i].OffHi\n\t})\n\n\tmaxExIdx := sort.Search(len(p.Extents), func(i int) bool {\n\t\treturn hi <= p.Extents[i].OffLo\n\t})\n\n\tif minExIdx > maxExIdx {\n\t\t// this can happen if lo > hi.\n\t\t// (basically a programmer error)\n\t\tmaxExIdx = minExIdx\n\t}\n\n\treturn minExIdx, maxExIdx\n}\n\n// OccludesStream will tell you if the page's cached contents\n// fully occlude the underlying stream. Or in other words:\n// If true, we do not need to read from the underlying stream.\nfunc (p *Page) OccludesStream(pageOff, length uint32) bool {\n\tl := int64(length)\n\tminExIdx, maxExIdx := p.affectedExtentIdxs(pageOff, pageOff+length)\n\tfor idx := minExIdx; idx < maxExIdx && l > 0; idx++ {\n\t\tex := p.Extents[idx]\n\t\tif ex.OffHi < pageOff {\n\t\t\tcontinue\n\t\t}\n\n\t\tif ex.OffLo < pageOff {\n\t\t\tl -= int64(ex.OffHi - pageOff)\n\t\t\tcontinue\n\t\t}\n\n\t\tl -= int64(ex.OffHi - ex.OffLo)\n\t}\n\n\treturn l <= 0\n}\n\n// Overlay adds newly written data in `write` to the page\n// at `off` (relative to the page start!). off + len(write) may not\n// exceed the page size! This is a programmer error.\n//\n// Internally, the data is copied to the page buffer and we keep\n// note of the new data in an extent, possibly merging with existing\n// ones. This is a relatively fast operation.\nfunc (p *Page) Overlay(off uint32, write []byte) {\n\tif len(write) == 0 {\n\t\treturn\n\t}\n\n\toffPlusWrite := off + uint32(len(write))\n\tif offPlusWrite > uint32(len(p.Data)) {\n\t\t// this is a programmer error:\n\t\tpanic(fmt.Sprintf(\"extent with write over page bound: %d\", offPlusWrite))\n\t}\n\n\t// Copy the data to the requested part of the page.\n\t// Everything after is maintaining the extents.\n\tcopy(p.Data[off:offPlusWrite], write)\n\tp.updateExtents(off, offPlusWrite)\n}\n\nfunc (p *Page) updateExtents(off, offPlusWrite uint32) {\n\t// base case: no extents yet:\n\tif len(p.Extents) == 0 {\n\t\tp.Extents = append(p.Extents, Extent{\n\t\t\tOffLo: off,\n\t\t\tOffHi: offPlusWrite,\n\t\t})\n\t\treturn\n\t}\n\n\t// Find out where to insert the new extent.\n\t// Use binary search to find a range of extents\n\t// that are affected by this write.\n\tminExIdx, maxExIdx := p.affectedExtentIdxs(off, offPlusWrite)\n\n\tif minExIdx >= len(p.Extents) {\n\t\t// This means that no extent was affected because we wrote beyond any\n\t\t// existing extent. Append a new extent to the end of the list.\n\t\tp.Extents = append(p.Extents, Extent{\n\t\t\tOffLo: off,\n\t\t\tOffHi: offPlusWrite,\n\t\t})\n\t\treturn\n\t}\n\n\tif minExIdx == maxExIdx {\n\t\t// write happens in \"free space\". No extent hit.\n\t\tif minExIdx > 0 && p.Extents[minExIdx-1].OffHi == off {\n\t\t\t// If the write happens to be right after another existing extent\n\t\t\t// then merge with it. Otherwise insert below.\n\t\t\tp.Extents[minExIdx-1].OffHi = offPlusWrite\n\t\t\treturn\n\t\t}\n\n\t\tif maxExIdx < len(p.Extents) && p.Extents[maxExIdx].OffLo == offPlusWrite {\n\t\t\t// If the write happens to be right before another existing extent\n\t\t\t// then merge with it. Otherwise insert below.\n\t\t\tp.Extents[maxExIdx].OffLo = off\n\t\t\treturn\n\t\t}\n\n\t\t// insert new extent in the middle of the slice.\n\t\tp.Extents = append(p.Extents, Extent{})\n\t\tcopy(p.Extents[minExIdx+1:], p.Extents[minExIdx:])\n\t\tp.Extents[minExIdx] = Extent{\n\t\t\tOffLo: off,\n\t\t\tOffHi: offPlusWrite,\n\t\t}\n\n\t\treturn\n\t}\n\n\t// Join all affected in the range to one single extent,\n\t// and move rest of extents further and cut to new size:\n\tnewHi := p.Extents[maxExIdx-1].OffHi\n\tnewLo := p.Extents[minExIdx].OffLo\n\tif newHi < offPlusWrite {\n\t\tnewHi = offPlusWrite\n\t}\n\n\tif newLo > off {\n\t\tnewLo = off\n\t}\n\n\tp.Extents[minExIdx].OffLo = newLo\n\tp.Extents[minExIdx].OffHi = newHi\n\tcopy(p.Extents[minExIdx+1:], p.Extents[maxExIdx:])\n\tp.Extents = p.Extents[:len(p.Extents)-(maxExIdx-minExIdx)+1]\n}\n\nfunc minUint32(a, b uint32) uint32 {\n\tif a < b {\n\t\treturn a\n\t}\n\n\treturn b\n}\n\n// Underlay is like the \"negative\" of Overlay. It writes the data of `write`\n// (starting at pageOff) to the underlying buffer where *no* extent is.\n// It can be used to \"cache\" data from the underlying stream, but not\n// overwriting any overlay. If OccludesStream() returns true for the same\n// offsets, then Underlay() will be an (expensive) no-op.\nfunc (p *Page) Underlay(pageOff uint32, write []byte) {\n\tpageOffPlusWrite := pageOff + uint32(len(write))\n\tif pageOff == pageOffPlusWrite {\n\t\t// zero underlay.\n\t\treturn\n\t}\n\n\tcursor := write\n\tprevOff := pageOff\n\tfor _, ex := range p.Extents {\n\t\tif ex.OffHi < pageOff {\n\t\t\t// Extent was before the desired write.\n\t\t\t// No need to consider this one.\n\t\t\tcontinue\n\t\t}\n\n\t\tif ex.OffLo < pageOff {\n\t\t\t// Extent started before pageOff,\n\t\t\t// but goes over it. We should not copy.\n\t\t\t// Instead \"loose\" the data of that extent.\n\t\t\tcutoff := minUint32(ex.OffHi-pageOff, uint32(len(cursor)))\n\t\t\tcursor = cursor[cutoff:]\n\t\t\tprevOff = ex.OffHi\n\t\t\tcontinue\n\t\t}\n\n\t\ttoCopy := ex.OffLo - prevOff\n\t\tif toCopy > 0 {\n\t\t\t// Copy everything since last copy\n\t\t\t// to p.Data and jump over the data in cursor.\n\t\t\tcopy(p.Data[prevOff:prevOff+toCopy], cursor)\n\t\t}\n\n\t\tcursor = cursor[minUint32(toCopy+ex.OffHi-ex.OffLo, uint32(len(cursor))):]\n\t\tprevOff = ex.OffHi\n\t}\n\n\tif prevOff < pageOffPlusWrite && len(cursor) > 0 {\n\t\t// Handle the case when the underlying write\n\t\t// goes beyond all extents or when there are\n\t\t// no extents at all.\n\t\ttoCopy := pageOffPlusWrite - prevOff\n\t\tcopy(p.Data[prevOff:prevOff+toCopy], cursor)\n\t}\n\n\tp.updateExtents(pageOff, pageOffPlusWrite)\n}\n" }, { "path": "catfs/mio/pagecache/page/page_test.go", "content": "package page\n\nimport (\n\t\"testing\"\n\n\t\"github.com/sahib/brig/util/testutil\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestPageAffectedIndices(t *testing.T) {\n\te1 := Extent{\n\t\tOffLo: 0,\n\t\tOffHi: Size / 4,\n\t}\n\n\te2 := Extent{\n\t\tOffLo: 2 * Size / 4,\n\t\tOffHi: 3 * Size / 4,\n\t}\n\n\te3 := Extent{\n\t\tOffLo: 3 * Size / 4,\n\t\tOffHi: 4 * Size / 4,\n\t}\n\n\tp := Page{\n\t\tData: make([]byte, Size+Meta),\n\t\tExtents: []Extent{e1, e2, e3},\n\t}\n\n\tfor idx := uint32(1); idx < e1.OffHi; idx++ {\n\t\tl, h := p.affectedExtentIdxs(0, uint32(idx))\n\n\t\t// result means: affects extents[0] until\n\t\t// (excluding) extents[1]\n\t\trequire.Equal(t, 0, l, idx)\n\t\trequire.Equal(t, 1, h, idx)\n\t}\n\n\tfor idx := e1.OffHi; idx < e2.OffLo; idx++ {\n\t\tl, h := p.affectedExtentIdxs(e1.OffHi, uint32(idx))\n\n\t\t// result means: no extent was found, the array\n\t\t// of affected extents is empty.\n\t\trequire.Equal(t, 1, l, idx)\n\t\trequire.Equal(t, 1, h, idx)\n\t}\n\n\tfor idx := e2.OffLo + 1; idx < e3.OffLo; idx++ {\n\t\tl, h := p.affectedExtentIdxs(idx, idx+1)\n\n\t\t// result means: no extent was found, the array\n\t\t// of affected extents is empty.\n\t\trequire.Equal(t, 1, l, idx)\n\t\trequire.Equal(t, 2, h, idx)\n\t}\n\n\t// No for loop needed for last case:\n\tl, h := p.affectedExtentIdxs(Size, Size+1)\n\trequire.Equal(t, 3, l)\n\trequire.Equal(t, 3, h)\n}\n\nfunc TestPageSerializeDeserialize(t *testing.T) {\n\texpected := New(0, testutil.CreateDummyBuf(Size))\n\n\tdata := expected.AsBytes()\n\tgot, err := FromBytes(data)\n\trequire.NoError(t, err)\n\n\trequire.Equal(t, expected.Data, got.Data)\n\trequire.Equal(t, expected.Extents, got.Extents)\n}\n\nfunc TestPageSerializeWithManyWrites(t *testing.T) {\n\t// Simulate a really pathological case where we have tons of small writes.\n\t// They will cause AsBytes() to increase the backing buffer when\n\t// serializing. This will be a performance hit, but is at least correct.\n\texpected := New(0, testutil.CreateDummyBuf(10))\n\tfor idx := 0; idx < 1024; idx++ {\n\t\toff := uint32(11 + (idx * 11))\n\t\texpected.Overlay(off, testutil.CreateDummyBuf(10))\n\t}\n\n\t// 1025: initial extent existed:\n\trequire.Len(t, expected.Extents, 1025)\n\n\tdata := expected.AsBytes()\n\tgot, err := FromBytes(data)\n\trequire.NoError(t, err)\n\n\trequire.Equal(t, expected.Data, got.Data)\n\trequire.Equal(t, expected.Extents, got.Extents)\n}\n\nfunc TestPageOccludeStreamBasic(t *testing.T) {\n\t// page with one extent:\n\tp := New(0, testutil.CreateDummyBuf(Size/4))\n\n\trequire.False(t, p.OccludesStream(0, Size))\n\trequire.False(t, p.OccludesStream(0, Size/4+1))\n\trequire.True(t, p.OccludesStream(0, Size/4))\n\trequire.True(t, p.OccludesStream(0, Size/4-1))\n\n\tp.Overlay(2*Size/4, testutil.CreateDummyBuf(Size/4))\n\n\trequire.False(t, p.OccludesStream(2*Size/4, Size/4+1))\n\trequire.False(t, p.OccludesStream(0, 3*Size/4+1))\n\trequire.True(t, p.OccludesStream(2*Size/4, Size/4))\n\trequire.True(t, p.OccludesStream(2*Size/4, Size/4-1))\n}\n\nfunc TestPageOccludeStreamInExtent(t *testing.T) {\n\tp := New(0, testutil.CreateDummyBuf(Size/4))\n\tp.Overlay(2*Size/4, testutil.CreateDummyBuf(Size/4))\n\n\trequire.False(t, p.OccludesStream(Size/8, 5*Size/8))\n\trequire.False(t, p.OccludesStream(5*Size/8, Size/8+1))\n\trequire.True(t, p.OccludesStream(5*Size/8, Size/8))\n\trequire.True(t, p.OccludesStream(Size/8, Size/8))\n}\n\nfunc TestPageAddExtent(t *testing.T) {\n\t// page with one extent:\n\tp := New(0, testutil.CreateDummyBuf(Size/4))\n\n\t// This matches the extents in TestPageAffectedIndices:\n\t// (first extent touches existing one!)\n\tp.Overlay(100, testutil.CreateDummyBuf(Size/4))\n\tp.Overlay(2*Size/4, testutil.CreateDummyBuf(Size/4))\n\tp.Overlay(3*Size/4, testutil.CreateDummyBuf(Size/4))\n\n\trequire.Len(t, p.Extents, 3)\n\trequire.Equal(\n\t\tt,\n\t\t[]Extent{{0, Size/4 + 100}, {2 * Size / 4, 3 * Size / 4}, {3 * Size / 4, Size}},\n\t\tp.Extents,\n\t)\n\n\trequire.Panics(t, func() {\n\t\t// Write beyond the extents:\n\t\tp.Overlay(Size, testutil.CreateDummyBuf(1))\n\t})\n\n\t// Write an extent in free space, right after another one:\n\t// It should detect this and merge with it.\n\tp.Overlay(Size/4+100, testutil.CreateDummyBuf(20))\n\trequire.Len(t, p.Extents, 3)\n\trequire.Equal(t, Extent{0, Size/4 + 120}, p.Extents[0])\n\n\t// Write an extent in free space (not adjacent):\n\tp.Overlay(Size/4+200, testutil.CreateDummyBuf(30))\n\trequire.Len(t, p.Extents, 4)\n\trequire.Equal(t, Extent{Size/4 + 200, Size/4 + 230}, p.Extents[1])\n\n\t// Write an extent that covers everything,\n\t// should reduce to a single one:\n\tp.Overlay(0, testutil.CreateDummyBuf(Size))\n\trequire.Len(t, p.Extents, 1)\n\trequire.Equal(t, Extent{0, Size}, p.Extents[0])\n\trequire.Equal(t, testutil.CreateDummyBuf(Size), p.Data)\n\n\t// Try to add an empty extent, it should not do anything.\n\tp.Overlay(0, []byte{})\n\trequire.Len(t, p.Extents, 1)\n}\n\nfunc TestPageAddExtentRegression(t *testing.T) {\n\t// I forgot to adjust the lower extent bound:\n\tp := New(3*Size/4, testutil.CreateDummyBuf(Size/4))\n\tp.Overlay(0, testutil.CreateDummyBuf(Size))\n\trequire.Len(t, p.Extents, 1)\n\trequire.Equal(t, uint32(0), p.Extents[0].OffLo)\n\trequire.Equal(t, uint32(Size), p.Extents[0].OffHi)\n}\n\nfunc TestPageUnderlayFull(t *testing.T) {\n\tunderlay := testutil.CreateRandomDummyBuf(Size, 23)\n\toverlay := testutil.CreateDummyBuf(Size / 4)\n\n\tp := New(2*Size/4, overlay)\n\tp.Underlay(0, underlay)\n\n\tcopy(underlay[2*Size/4:], overlay)\n\trequire.Equal(t, len(underlay), len(p.Data))\n\trequire.Equal(t, underlay, p.Data)\n}\n\nfunc TestPageUnderlayPartial(t *testing.T) {\n\tunderlay := testutil.CreateRandomDummyBuf(2*Size/4, 23)\n\tp := New(2*Size/4, testutil.CreateDummyBuf(Size/4))\n\n\t// That overlay should be ignored:\n\tp.Overlay(Size/16, testutil.CreateDummyBuf(Size/32))\n\n\t// This overlay shadows the underlay:\n\tp.Overlay(Size/8, testutil.CreateDummyBuf(Size/4))\n\n\t// Now underlay it. Should only write things\n\t// between 3*Size/8 and Size/2, everything else is shadowed.\n\tp.Underlay(Size/4, underlay)\n\n\t// Construct our expectation:\n\texpected := make([]byte, Size)\n\tcopy(expected[Size/4:], underlay)\n\tcopy(expected[Size/16:], testutil.CreateDummyBuf(Size/32))\n\tcopy(expected[Size/8:], testutil.CreateDummyBuf(Size/4))\n\tcopy(expected[2*Size/4:], testutil.CreateDummyBuf(Size/4))\n\n\trequire.Equal(t, expected, p.Data)\n}\n\nfunc TestPageUnderlayLeftover(t *testing.T) {\n\tunderlay := testutil.CreateRandomDummyBuf(1*Size/4, 23)\n\toverlay := testutil.CreateDummyBuf(3 * Size / 4)\n\tp := New(0, overlay)\n\n\t// should do nothing!\n\tp.Underlay(0, underlay)\n\n\t// Construct our expectation:\n\texpected := make([]byte, Size)\n\tcopy(expected, overlay)\n\trequire.Equal(t, expected, p.Data)\n}\n" }, { "path": "catfs/mio/pagecache/util.go", "content": "package pagecache\n\nimport (\n\t\"io\"\n)\n\n// small util to wrap a buffer we want to write to. Tells you easily how much\n// data you can still write to it.\ntype iobuf struct {\n\tdst []byte\n\toff int\n}\n\nfunc (ib *iobuf) Write(src []byte) (int, error) {\n\tn := copy(ib.dst[ib.off:ib.off+ib.Left()], src)\n\tib.off += n\n\treturn n, nil\n}\n\nfunc (ib *iobuf) Len() int {\n\treturn ib.off\n}\n\nfunc (ib *iobuf) Left() int {\n\treturn len(ib.dst) - ib.off\n}\n\n// zeroPadReader wraps another reader which has data\n// until `size`. If `length` > `size` than it pads the\n// gap with zero reads.\ntype zeroPadReader struct {\n\tr io.Reader\n\toff, size, length int64\n}\n\nfunc memzero(buf []byte) {\n\t// NOTE: This for-loop is optimized by the compiler:\n\t// https://github.com/golang/go/issues/5373\n\t//\n\t// (copy with a pre-allocated zero page is 2x slower than this!)\n\tfor idx := range buf {\n\t\tbuf[idx] = 0\n\t}\n}\n\nfunc (zpr *zeroPadReader) Read(buf []byte) (int, error) {\n\tif zpr.size >= zpr.length {\n\t\t// sanity check. zpr.length might be also shorter.\n\t\t// then we don't do any padding but work like\n\t\t// io.LimitReader().\n\t\tzpr.size = zpr.length\n\t}\n\n\tdiff := zpr.length - zpr.off\n\tbufLen := int64(len(buf))\n\tif diff < bufLen {\n\t\t// clamp buf to zpr.length\n\t\tbufLen = diff\n\t}\n\n\tif zpr.off < zpr.size {\n\t\t// below underlying stream size:\n\t\tn, err := zpr.r.Read(buf[:bufLen])\n\t\tzpr.off += int64(n)\n\t\treturn n, err\n\t}\n\n\tif diff > 0 {\n\t\t// above underlying stream size,\n\t\t// but below padded length.\n\t\tmemzero(buf[:bufLen])\n\t\tzpr.off += bufLen\n\t\treturn int(bufLen), nil\n\t}\n\n\treturn 0, io.EOF\n}\n\n/////////\n\n// copyNBuffer is golang's io.CopyN with added param for the buffer,\n// like in io.CopyBuffer. Saves precious allocations.\nfunc copyNBuffer(dst io.Writer, src io.Reader, n int64, buf []byte) (written int64, err error) {\n\twritten, err = io.CopyBuffer(dst, io.LimitReader(src, n), buf)\n\tif written == n {\n\t\treturn n, nil\n\t}\n\n\tif written < n && err == nil {\n\t\t// src stopped early; must have been EOF.\n\t\terr = io.EOF\n\t}\n\n\treturn\n}\n" }, { "path": "catfs/mio/pagecache/util_test.go", "content": "package pagecache\n\nimport (\n\t\"bytes\"\n\t\"io\"\n\t\"testing\"\n\n\t\"github.com/sahib/brig/util\"\n\t\"github.com/sahib/brig/util/testutil\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestZeroPaddedReader(t *testing.T) {\n\ttcs := []struct {\n\t\tname string\n\t\toff, length, size int64\n\t}{\n\t\t{\n\t\t\tname: \"usual-case\",\n\t\t\toff: 0,\n\t\t\tlength: 1024,\n\t\t\tsize: 512,\n\t\t}, {\n\t\t\tname: \"truncate-short\",\n\t\t\toff: 0,\n\t\t\tlength: 512,\n\t\t\tsize: 1024,\n\t\t}, {\n\t\t\tname: \"equal\",\n\t\t\toff: 0,\n\t\t\tlength: 1024,\n\t\t\tsize: 1024,\n\t\t}, {\n\t\t\tname: \"zero\",\n\t\t\toff: 0,\n\t\t\tlength: 0,\n\t\t\tsize: 0,\n\t\t},\n\t}\n\n\tconst maxSize = 4 * 1024\n\tdata := testutil.CreateDummyBuf(maxSize)\n\tzero := make([]byte, maxSize)\n\n\tfor _, tc := range tcs {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tzpr := &zeroPadReader{\n\t\t\t\tr: bytes.NewReader(data),\n\t\t\t\toff: tc.off,\n\t\t\t\tsize: tc.size,\n\t\t\t\tlength: tc.length,\n\t\t\t}\n\n\t\t\tout := &bytes.Buffer{}\n\t\t\tn, err := io.Copy(out, zpr)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Equal(t, tc.length, n)\n\n\t\t\ta := util.Min64(tc.size, tc.length)\n\t\t\tb := util.Max64(tc.size, tc.length)\n\n\t\t\toutData := out.Bytes()\n\t\t\trequire.Equal(t, data[0:a], outData[0:a])\n\t\t\trequire.Equal(t, zero[a:b], outData[a:b])\n\t\t})\n\t}\n}\n\nfunc TestIOBuf(t *testing.T) {\n\ttcs := []struct {\n\t\tname string\n\t\tsrcSize int\n\t\tdstSize int\n\t}{\n\t\t{\n\t\t\tname: \"src=dst\",\n\t\t\tsrcSize: 1024,\n\t\t\tdstSize: 1024,\n\t\t}, {\n\t\t\tname: \"srcdst\",\n\t\t\tsrcSize: 2048,\n\t\t\tdstSize: 512,\n\t\t}, {\n\t\t\tname: \"zero\",\n\t\t\tsrcSize: 0,\n\t\t\tdstSize: 0,\n\t\t},\n\t}\n\n\tfor _, tc := range tcs {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tsrc := testutil.CreateDummyBuf(int64(tc.srcSize))\n\t\t\tib := &iobuf{\n\t\t\t\tdst: make([]byte, tc.dstSize),\n\t\t\t}\n\n\t\t\tn, err := ib.Write(src)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Equal(t, n, ib.Len())\n\n\t\t\tmin := tc.srcSize\n\t\t\tif min > tc.dstSize {\n\t\t\t\tmin = tc.dstSize\n\t\t\t}\n\n\t\t\trequire.Equal(t, n, min)\n\t\t})\n\t}\n}\n\nfunc BenchmarkZeroing(b *testing.B) {\n\tb.Run(\"memzero\", func(b *testing.B) {\n\t\tbuf := testutil.CreateDummyBuf(16 * 1024)\n\t\tfor idx := 0; idx < b.N; idx++ {\n\t\t\tmemzero(buf)\n\t\t}\n\t})\n\n\tb.Run(\"zerocopy\", func(b *testing.B) {\n\t\tzero := make([]byte, 16*1024)\n\t\tbuf := testutil.CreateDummyBuf(16 * 1024)\n\t\tfor idx := 0; idx < b.N; idx++ {\n\t\t\tcopy(buf, zero)\n\t\t}\n\t})\n}\n" }, { "path": "catfs/mio/stream.go", "content": "package mio\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\n\t\"github.com/sahib/brig/catfs/mio/compress\"\n\t\"github.com/sahib/brig/catfs/mio/encrypt\"\n\t\"github.com/sahib/brig/repo/hints\"\n\t\"github.com/sahib/brig/util\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// Stream is a stream coming from the backend.\ntype Stream interface {\n\tio.Reader\n\tio.Seeker\n\tio.Closer\n\tio.WriterTo\n}\n\ntype stream struct {\n\tio.Reader\n\tio.Seeker\n\tio.Closer\n\tio.WriterTo\n}\n\ntype dumbWriterTo struct {\n\tr io.Reader\n}\n\nfunc (d dumbWriterTo) WriteTo(w io.Writer) (n int64, err error) {\n\treturn io.Copy(w, d.r)\n}\n\n// NewOutStream creates an OutStream piping data from brig to the outside.\n// `key` is used to decrypt the data. The compression algorithm is read\n// from the stream header.\nfunc NewOutStream(r io.ReadSeeker, isRaw bool, key []byte) (Stream, error) {\n\ts := stream{\n\t\tReader: r,\n\t\tSeeker: r,\n\t\tWriterTo: dumbWriterTo{r: r},\n\t\tCloser: ioutil.NopCloser(r),\n\t}\n\n\tif isRaw {\n\t\t// directly return stream.\n\t\treturn s, nil\n\t}\n\n\t// At this point we're sure that there must be a magic number.\n\t// We can use it to decide what readers we should build.\n\n\tmagicNumber, headerReader, err := util.PeekHeader(r, 8)\n\tif err != nil {\n\t\t// First read on the stream, errors will bubble up here.\n\t\treturn nil, err\n\t}\n\n\t// make sure that the header is prefixed to the stream again:\n\t// compress + encrypt reader expect the magic number there.\n\ts.Reader = headerReader\n\ts.Seeker = headerReader\n\ts.WriterTo = dumbWriterTo{r: headerReader}\n\n\t// NOTE: Assumption here is that our own magic numbers\n\t// are always 8 bytes long. Since we control it,\n\t// that's reasonable.\n\tif len(magicNumber) != 8 {\n\t\treturn nil, fmt.Errorf(\"bad magic number\")\n\t}\n\n\tvar isEncrypted bool\n\n\tswitch mn := string(magicNumber); mn {\n\tcase string(encrypt.MagicNumber):\n\t\tisEncrypted = true\n\tcase string(compress.MagicNumber):\n\t\t// Not encrypted, but decompress needed.\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"unknown magic number '%s'\", mn)\n\t}\n\n\tif isEncrypted {\n\t\trEnc, err := encrypt.NewReader(s, key)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tflags, err := rEnc.Flags()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\ts.Reader = rEnc\n\t\ts.Seeker = rEnc\n\t\ts.WriterTo = rEnc\n\n\t\t// The encryption header stores if we encoded the stream\n\t\t// with another stream inside (matroska like). If not,\n\t\t// we can return early.\n\t\tif flags&encrypt.FlagCompressedInside == 0 {\n\t\t\treturn s, nil\n\t\t}\n\t}\n\n\t// if compression is used inside, than wrap in decompressor:\n\t// (s might contain decryptor or is raw stream)\n\trZip := compress.NewReader(s)\n\ts.Reader = rZip\n\ts.Seeker = rZip\n\ts.WriterTo = rZip\n\treturn s, nil\n}\n\nfunc guessCompression(path string, r io.Reader, hint *hints.Hint) (io.Reader, error) {\n\t// Keep the header of the file in memory, so we can do some guessing\n\t// of e.g. the compression algorithm we should use.\n\theaderReader := util.NewHeaderReader(r, 2048)\n\theaderBuf, err := headerReader.Peek()\n\tif err != nil {\n\t\tlog.WithError(err).Warnf(\"failed to peek stream header\")\n\t\treturn nil, err\n\t}\n\n\tcompressAlgo, err := compress.GuessAlgorithm(path, headerBuf)\n\tif err != nil {\n\t\t// NOTE: don't error out here. That just means we don't\n\t\t// guessed the perfect settings.\n\t\tlog.WithError(err).\n\t\t\tWithField(\"path\", path).\n\t\t\tWarnf(\"failed to guess suitable zip algorithm\")\n\t}\n\n\tlog.Debugf(\"guessed '%s' compression for file %s\", compressAlgo, path)\n\thint.CompressionAlgo = hints.CompressAlgorithmTypeToCompressionHint(compressAlgo)\n\treturn headerReader, nil\n}\n\n// NewInStream creates a new stream that pipes data into ipfs.\n// The data is read from `r`, encrypted with `key` and encoded based on the\n// settings given by `hint`. `path` is only used to better guess the compression\n// algorithm - if desired by `hint`. `path` can be empty.\n//\n// It returns a reader that will produce the encoded stream.\n// If no actual encoding will be done, the second return param will be true\nfunc NewInStream(r io.Reader, path string, key []byte, hint hints.Hint) (io.ReadCloser, bool, error) {\n\tvar err error\n\n\tif hint.CompressionAlgo == hints.CompressionGuess {\n\t\t// replace \"guess\" to an actual compression algorithm.\n\t\tr, err = guessCompression(path, r, &hint)\n\t\tif err != nil {\n\t\t\treturn nil, false, err\n\t\t}\n\t}\n\n\t// use a pipe to redirect `r` to encoding writers without copying:\n\tpr, pw := io.Pipe()\n\n\t// Writing to pw will be matched by a read on the other side.\n\t// If there is no read we will block.\n\tvar w io.Writer = pw\n\tvar closers = []io.Closer{pw}\n\n\t// Only add encryption if desired by hints:\n\tif hint.EncryptionAlgo != hints.EncryptionNone {\n\t\twEnc, err := encrypt.NewWriter(w, key, hint.EncryptFlags())\n\t\tif err != nil {\n\t\t\treturn nil, false, err\n\t\t}\n\n\t\tclosers = append(closers, wEnc)\n\t\tw = wEnc\n\t}\n\n\t// Only add compression if desired or mime type is suitable:\n\tif hint.CompressionAlgo != hints.CompressionNone {\n\t\twZip, err := compress.NewWriter(w, hint.CompressionAlgo.ToCompressAlgorithmType())\n\t\tif err != nil {\n\t\t\treturn nil, false, err\n\t\t}\n\n\t\tclosers = append(closers, wZip)\n\t\tw = wZip\n\t}\n\n\t// Suck the reader empty and move it to `w`.\n\tgo func() {\n\t\tif _, err := io.Copy(w, r); err != nil {\n\t\t\t// Continue closing the fds; no return.\n\t\t\tlog.WithError(err).Warnf(\"internal write error\")\n\t\t}\n\n\t\t// NOTE: closers must be closed in inverse order.\n\t\t// pipe writer should come last. Each Close()\n\t\t// might still write out data.\n\t\tfor idx := len(closers) - 1; idx >= 0; idx-- {\n\t\t\tif err := closers[idx].Close(); err != nil {\n\t\t\t\tlog.WithError(err).Warnf(\"internal close error\")\n\t\t\t}\n\t\t}\n\t}()\n\n\treturn pr, hint.IsRaw(), nil\n}\n\n// limitedStream is a small wrapper around Stream,\n// which allows truncating the stream at a certain size.\ntype limitedStream struct {\n\tstream Stream\n\tpos uint64\n\tsize uint64\n}\n\nfunc (ls *limitedStream) Read(buf []byte) (int, error) {\n\tisEOF := false\n\tif ls.pos+uint64(len(buf)) >= ls.size {\n\t\tbuf = buf[:ls.size-ls.pos]\n\t\tisEOF = true\n\t}\n\n\tn, err := ls.stream.Read(buf)\n\tif err != nil {\n\t\treturn n, err\n\t}\n\n\tif isEOF {\n\t\terr = io.EOF\n\t}\n\n\treturn n, err\n}\n\nfunc (ls *limitedStream) Seek(offset int64, whence int) (int64, error) {\n\tswitch whence {\n\tcase io.SeekCurrent:\n\t\treturn ls.Seek(int64(ls.pos)+offset, io.SeekStart)\n\tcase io.SeekEnd:\n\t\tls.pos = 0\n\t\treturn ls.Seek(int64(ls.size)+offset, io.SeekStart)\n\tcase io.SeekStart:\n\t\tls.pos = 0\n\t}\n\n\tnewPos := int64(ls.pos) + offset\n\tif newPos < 0 {\n\t\treturn -1, io.EOF\n\t}\n\n\tif newPos > int64(ls.size) {\n\t\treturn int64(ls.size), io.EOF\n\t}\n\n\tls.pos = uint64(newPos)\n\treturn ls.stream.Seek(newPos, io.SeekStart)\n}\n\nfunc (ls *limitedStream) WriteTo(w io.Writer) (int64, error) {\n\t// We do not want to defeat the purpose of WriteTo here.\n\t// That's why we do the limit check in the writer part.\n\treturn ls.stream.WriteTo(util.LimitWriter(w, int64(ls.size-ls.pos)))\n}\n\nfunc (ls *limitedStream) Close() error {\n\treturn ls.stream.Close()\n}\n\n// LimitStream is like io.LimitReader, but works for mio.Stream.\n// It will not allow reading/seeking after the specified size.\nfunc LimitStream(stream Stream, size uint64) Stream {\n\treturn &limitedStream{\n\t\tstream: stream,\n\t\tpos: 0,\n\t\tsize: size,\n\t}\n}\n" }, { "path": "catfs/mio/stream_test.go", "content": "package mio\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"testing\"\n\n\t\"github.com/brianvoe/gofakeit/v6\"\n\t\"github.com/sahib/brig/catfs/mio/compress\"\n\t\"github.com/sahib/brig/repo/hints\"\n\t\"github.com/sahib/brig/util/testutil\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nvar testKey = []byte(\"01234567890ABCDE01234567890ABCDE\")\n\nfunc testWriteAndRead(\n\tt *testing.T,\n\traw []byte,\n\thint hints.Hint,\n) {\n\trawBuf := &bytes.Buffer{}\n\tif _, err := rawBuf.Write(raw); err != nil {\n\t\tt.Errorf(\"Huh, buf-write failed?\")\n\t\treturn\n\t}\n\n\tencStream, isRaw, err := NewInStream(\n\t\trawBuf,\n\t\tgofakeit.Name(),\n\t\ttestKey,\n\t\thint,\n\t)\n\tif err != nil {\n\t\tt.Errorf(\"creating encryption stream failed: %v\", err)\n\t\treturn\n\t}\n\n\tencrypted := &bytes.Buffer{}\n\tif _, err = io.Copy(encrypted, encStream); err != nil {\n\t\tt.Errorf(\"reading encrypted data failed: %v\", err)\n\t\treturn\n\t}\n\n\t// Fake a close method:\n\tbr := bytes.NewReader(encrypted.Bytes())\n\n\tr := stream{\n\t\tReader: br,\n\t\tSeeker: br,\n\t\tWriterTo: br,\n\t\tCloser: ioutil.NopCloser(nil),\n\t}\n\n\tdecStream, err := NewOutStream(r, isRaw, testKey)\n\tif err != nil {\n\t\tt.Errorf(\"creating decryption stream failed: %v\", err)\n\t\treturn\n\t}\n\n\tdecrypted := &bytes.Buffer{}\n\tif _, err = io.Copy(decrypted, decStream); err != nil {\n\t\tt.Errorf(\"reading decrypted data failed: %v\", err)\n\t\treturn\n\t}\n\n\tif !bytes.Equal(decrypted.Bytes(), raw) {\n\t\tt.Errorf(\"Raw and decrypted is not equal => BUG.\")\n\t\tt.Errorf(\"RAW:\\n %v\", raw)\n\t\tt.Errorf(\"DEC:\\n %v\", decrypted.Bytes())\n\t\treturn\n\t}\n}\n\nfunc TestWriteAndRead(t *testing.T) {\n\tt.Parallel()\n\n\ts64k := int64(64 * 1024)\n\tsizes := []int64{\n\t\t0,\n\t\t1,\n\t\t10,\n\t\ts64k,\n\t\ts64k - 1,\n\t\ts64k + 1,\n\t\ts64k * 2,\n\t\ts64k * 1024,\n\t}\n\n\tfor _, size := range sizes {\n\t\tregularData := testutil.CreateDummyBuf(size)\n\t\trandomData := testutil.CreateRandomDummyBuf(size, 42)\n\n\t\tfor _, hint := range hints.AllPossibleHints() {\n\t\t\tprefix := fmt.Sprintf(\"%s-size-%d-\", hint, size)\n\t\t\tt.Run(prefix+\"regular\", func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\t\t\t\ttestWriteAndRead(t, regularData, hint)\n\t\t\t})\n\t\t\tt.Run(prefix+\"random\", func(t *testing.T) {\n\t\t\t\tt.Parallel()\n\t\t\t\ttestWriteAndRead(t, randomData, hint)\n\t\t\t})\n\t\t}\n\t}\n}\n\nfunc TestLimitedStream(t *testing.T) {\n\tt.Parallel()\n\n\ttestData := []byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}\n\n\tr := bytes.NewReader(testData)\n\n\t// Fake a stream without all the encryption/compression fuzz.\n\tstream := struct {\n\t\tio.Reader\n\t\tio.Seeker\n\t\tio.Closer\n\t\tio.WriterTo\n\t}{\n\t\tReader: r,\n\t\tSeeker: r,\n\t\tWriterTo: r,\n\t\tCloser: ioutil.NopCloser(r),\n\t}\n\n\tfor idx := 0; idx <= 10; idx++ {\n\t\t// Seek back to beginning:\n\t\t_, err := stream.Seek(0, io.SeekStart)\n\t\trequire.Nil(t, err)\n\n\t\tsmallStream := LimitStream(stream, uint64(idx))\n\t\tdata, err := ioutil.ReadAll(smallStream)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, testData[:idx], data)\n\t}\n\n\tvar err error\n\n\t// Reset and do some special torturing:\n\t_, err = stream.Seek(0, io.SeekStart)\n\trequire.Nil(t, err)\n\n\tlimitStream := LimitStream(stream, 5)\n\n\tn, err := limitStream.Seek(4, io.SeekStart)\n\trequire.Nil(t, err)\n\trequire.Equal(t, int64(4), n)\n\n\tn, err = limitStream.Seek(6, io.SeekStart)\n\trequire.True(t, err == io.EOF)\n\n\tn, err = limitStream.Seek(-5, io.SeekEnd)\n\trequire.Nil(t, err)\n\trequire.Equal(t, int64(0), n)\n\n\t_, err = limitStream.Seek(-6, io.SeekEnd)\n\trequire.True(t, err == io.EOF)\n\n\t_, err = stream.Seek(0, io.SeekStart)\n\trequire.Nil(t, err)\n\n\tlimitStream = LimitStream(stream, 5)\n\n\tbuf := &bytes.Buffer{}\n\tn, err = limitStream.WriteTo(buf)\n\trequire.Nil(t, err)\n\trequire.Equal(t, n, int64(10))\n\trequire.Equal(t, buf.Bytes(), testData[:5])\n\n\tbuf.Reset()\n\t_, err = stream.Seek(0, io.SeekStart)\n\trequire.Nil(t, err)\n\tlimitStream = LimitStream(stream, 11)\n\n\tn, err = limitStream.WriteTo(buf)\n\trequire.Nil(t, err)\n\trequire.Equal(t, n, int64(10))\n\trequire.Equal(t, buf.Bytes(), testData)\n}\n\nfunc TestLimitStreamSize(t *testing.T) {\n\t// Size taken from a dummy file that showed this bug:\n\tdata := testutil.CreateDummyBuf(6041)\n\tpackData, err := compress.Pack(data, compress.AlgoSnappy)\n\trequire.Nil(t, err)\n\n\trZip := compress.NewReader(bytes.NewReader(packData))\n\tstream := struct {\n\t\tio.Reader\n\t\tio.Seeker\n\t\tio.Closer\n\t\tio.WriterTo\n\t}{\n\t\tReader: rZip,\n\t\tSeeker: rZip,\n\t\tWriterTo: rZip,\n\t\tCloser: ioutil.NopCloser(rZip),\n\t}\n\n\tr := LimitStream(stream, uint64(len(data)))\n\n\tsize, err := r.Seek(0, io.SeekEnd)\n\trequire.Nil(t, err)\n\trequire.Equal(t, int64(len(data)), size)\n\n\toff, err := r.Seek(0, io.SeekStart)\n\trequire.Nil(t, err)\n\trequire.Equal(t, int64(0), off)\n\n\tbuf := &bytes.Buffer{}\n\tn, err := io.Copy(buf, r)\n\trequire.Nil(t, err)\n\trequire.Equal(t, int64(len(data)), n)\n\trequire.Equal(t, data, buf.Bytes())\n}\n\nfunc TestStreamSizeBySeek(t *testing.T) {\n\tbuf := &bytes.Buffer{}\n\tdata := testutil.CreateDummyBuf(6041 * 1024)\n\tencStream, isRaw, err := NewInStream(\n\t\tbytes.NewReader(data),\n\t\t\"\",\n\t\ttestKey,\n\t\thints.Default(),\n\t)\n\trequire.Nil(t, err)\n\n\t_, err = io.Copy(buf, encStream)\n\trequire.Nil(t, err)\n\n\tstream, err := NewOutStream(\n\t\tbytes.NewReader(buf.Bytes()),\n\t\tisRaw,\n\t\ttestKey,\n\t)\n\trequire.Nil(t, err)\n\n\tn, err := stream.Seek(0, io.SeekEnd)\n\trequire.Nil(t, err)\n\trequire.Equal(t, int64(len(data)), n)\n\n\tn, err = stream.Seek(0, io.SeekStart)\n\trequire.Nil(t, err)\n\trequire.Equal(t, int64(0), n)\n\n\toutBuf := &bytes.Buffer{}\n\tn, err = io.Copy(outBuf, stream)\n\trequire.Nil(t, err)\n\trequire.Equal(t, int64(len(data)), n)\n\trequire.Equal(t, outBuf.Bytes(), data)\n}\n" }, { "path": "catfs/nodes/base.go", "content": "package nodes\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\te \"github.com/pkg/errors\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tcapnp_model \"github.com/sahib/brig/catfs/nodes/capnp\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\tcapnp \"zombiezen.com/go/capnproto2\"\n)\n\n// Base is a place that holds all common attributes of all Nodes.\n// It also defines some utility function that will be mixed into real nodes.\ntype Base struct {\n\t// Basename of this node\n\tname string\n\n\t// name of the user that last modified this node\n\tuser string\n\n\t// Hash of this node (might be empty)\n\ttree h.Hash\n\n\t// Pointer hash to the content in the backend\n\tbackend h.Hash\n\n\t// Content hash of this node\n\tcontent h.Hash\n\n\t// Last modification time of this node.\n\tmodTime time.Time\n\n\t// Type of this node\n\tnodeType NodeType\n\n\t// Unique identifier for this node\n\tinode uint64\n}\n\n// copyBase will copy all attributes from the base.\nfunc (b *Base) copyBase(inode uint64) Base {\n\treturn Base{\n\t\tname: b.name,\n\t\tuser: b.user,\n\t\ttree: b.tree.Clone(),\n\t\tcontent: b.content.Clone(),\n\t\tbackend: b.backend.Clone(),\n\t\tmodTime: b.modTime,\n\t\tnodeType: b.nodeType,\n\t\tinode: inode,\n\t}\n}\n\n// User returns the user that last modified this node.\nfunc (b *Base) User() string {\n\treturn b.user\n}\n\n// Name returns the name of this node (e.g. /a/b/c -> c)\n// The root directory will have the name empty string.\nfunc (b *Base) Name() string {\n\treturn b.name\n}\n\n// TreeHash returns the hash of this node.\nfunc (b *Base) TreeHash() h.Hash {\n\treturn b.tree\n}\n\n// ContentHash returns the content hash of this node.\nfunc (b *Base) ContentHash() h.Hash {\n\treturn b.content\n}\n\n// BackendHash returns the backend hash of this node.\nfunc (b *Base) BackendHash() h.Hash {\n\treturn b.backend\n}\n\n// Type returns the type of this node.\nfunc (b *Base) Type() NodeType {\n\treturn b.nodeType\n}\n\n// ModTime will return the last time this node's content\n// was modified. Metadata changes are not recorded.\nfunc (b *Base) ModTime() time.Time {\n\treturn b.modTime\n}\n\n// Inode will return a unique ID that is different for each node.\nfunc (b *Base) Inode() uint64 {\n\treturn b.inode\n}\n\n/////// UTILS /////////\n\nfunc (b *Base) setBaseAttrsToNode(capnode capnp_model.Node) error {\n\tmodTimeBin, err := b.modTime.MarshalBinary()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif err := capnode.SetModTime(string(modTimeBin)); err != nil {\n\t\treturn err\n\t}\n\tif err := capnode.SetTreeHash(b.tree); err != nil {\n\t\treturn err\n\t}\n\tif err := capnode.SetContentHash(b.content); err != nil {\n\t\treturn err\n\t}\n\tif err := capnode.SetBackendHash(b.backend); err != nil {\n\t\treturn err\n\t}\n\tif err := capnode.SetName(b.name); err != nil {\n\t\treturn err\n\t}\n\tif err := capnode.SetUser(b.user); err != nil {\n\t\treturn err\n\t}\n\n\tcapnode.SetInode(b.inode)\n\treturn nil\n}\n\nfunc (b *Base) parseBaseAttrsFromNode(capnode capnp_model.Node) error {\n\tvar err error\n\tb.name, err = capnode.Name()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tb.user, err = capnode.User()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tb.tree, err = capnode.TreeHash()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tb.content, err = capnode.ContentHash()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tb.backend, err = capnode.BackendHash()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tunparsedModTime, err := capnode.ModTime()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif err := b.modTime.UnmarshalBinary([]byte(unparsedModTime)); err != nil {\n\t\treturn err\n\t}\n\n\tswitch typ := capnode.Which(); typ {\n\tcase capnp_model.Node_Which_file:\n\t\tb.nodeType = NodeTypeFile\n\tcase capnp_model.Node_Which_directory:\n\t\tb.nodeType = NodeTypeDirectory\n\tcase capnp_model.Node_Which_commit:\n\t\tb.nodeType = NodeTypeCommit\n\tcase capnp_model.Node_Which_ghost:\n\t\t// Ghost set the nodeType themselves.\n\t\t// Ignore them here.\n\tdefault:\n\t\treturn fmt.Errorf(\"Bad capnp node type `%d`\", typ)\n\t}\n\n\tb.inode = capnode.Inode()\n\treturn nil\n}\n\nfunc prefixSlash(s string) string {\n\tif !strings.HasPrefix(s, \"/\") {\n\t\treturn \"/\" + s\n\t}\n\n\treturn s\n}\n\n/////////////////////////////////////////\n// MARSHAL HELPERS FOR ARBITRARY NODES //\n/////////////////////////////////////////\n\n// MarshalNode will convert any Node to a byte string\n// Use UnmarshalNode to load a Node from it again.\nfunc MarshalNode(nd Node) ([]byte, error) {\n\tmsg, err := nd.ToCapnp()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn msg.Marshal()\n}\n\n// UnmarshalNode will try to interpret data as a Node\nfunc UnmarshalNode(data []byte) (Node, error) {\n\tmsg, err := capnp.Unmarshal(data)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapNd, err := capnp_model.ReadRootNode(msg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn CapNodeToNode(capNd)\n}\n\n// CapNodeToNode converts a capnproto `capNd` to a normal `Node`.\nfunc CapNodeToNode(capNd capnp_model.Node) (Node, error) {\n\t// Find out the correct node struct to initialize.\n\tvar node Node\n\n\tswitch typ := capNd.Which(); typ {\n\tcase capnp_model.Node_Which_ghost:\n\t\tnode = &Ghost{}\n\tcase capnp_model.Node_Which_file:\n\t\tnode = &File{}\n\tcase capnp_model.Node_Which_directory:\n\t\tnode = &Directory{}\n\tcase capnp_model.Node_Which_commit:\n\t\tnode = &Commit{}\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"Bad capnp node type `%d`\", typ)\n\t}\n\n\tif err := node.FromCapnpNode(capNd); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn node, nil\n}\n\n//////////////////////////\n// GENERAL NODE HELPERS //\n//////////////////////////\n\n// Depth returns the depth of the node.\n// It does this by looking at the path separators.\n// The depth of \"/\" is defined as 0.\nfunc Depth(nd Node) int {\n\tpath := nd.Path()\n\tif path == \"/\" {\n\t\treturn 0\n\t}\n\n\tdepth := 0\n\tfor _, rn := range path {\n\t\tif rn == '/' {\n\t\t\tdepth++\n\t\t}\n\t}\n\n\treturn depth\n}\n\n// RemoveNode removes `nd` from it's parent directory using `lkr`.\n// Removing the root is a no-op.\nfunc RemoveNode(lkr Linker, nd Node) error {\n\tparDir, err := ParentDirectory(lkr, nd)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Cannot remove root:\n\tif parDir == nil {\n\t\treturn nil\n\t}\n\n\treturn parDir.RemoveChild(lkr, nd)\n}\n\n// ParentDirectory returns the parent directory of `nd`.\n// For the root it will return nil.\nfunc ParentDirectory(lkr Linker, nd Node) (*Directory, error) {\n\tpar, err := nd.Parent(lkr)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif par == nil {\n\t\treturn nil, nil\n\t}\n\n\tparDir, ok := par.(*Directory)\n\tif !ok {\n\t\treturn nil, ie.ErrBadNode\n\t}\n\n\treturn parDir, nil\n}\n\n// ContentHash returns the correct content hash for `nd`.\n// This also works for ghosts where the content hash is taken from the\n// underlying node (ghosts themselve have no content).\nfunc ContentHash(nd Node) (h.Hash, error) {\n\tswitch nd.Type() {\n\tcase NodeTypeDirectory, NodeTypeCommit, NodeTypeFile:\n\t\treturn nd.ContentHash(), nil\n\tcase NodeTypeGhost:\n\t\tghost, ok := nd.(*Ghost)\n\t\tif !ok {\n\t\t\treturn nil, e.Wrapf(ie.ErrBadNode, \"cannot convert to ghost\")\n\t\t}\n\n\t\tswitch ghost.OldNode().Type() {\n\t\tcase NodeTypeFile:\n\t\t\toldFile, err := ghost.OldFile()\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\n\t\t\treturn oldFile.ContentHash(), nil\n\t\tcase NodeTypeDirectory:\n\t\t\toldDirectory, err := ghost.OldDirectory()\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\n\t\t\treturn oldDirectory.ContentHash(), nil\n\t\t}\n\t}\n\n\treturn nil, ie.ErrBadNode\n}\n" }, { "path": "catfs/nodes/capnp/nodes.capnp", "content": "using Go = import \"/go.capnp\";\n\n@0x9195d073cb5c5953;\n\n$Go.package(\"capnp\");\n$Go.import(\"github.com/sahib/brig/catfs/nodes/capnp\");\n\nstruct Commit $Go.doc(\"Commit is a set of changes to nodes\") {\n # Following attributes will be part of the hash:\n message @0 :Text;\n author @1 :Text;\n parent @2 :Data; # Hash to parent.\n root @3 :Data; # Hash to root directory.\n index @4 :Int64; # Total number of commits.\n\n # Attributes not being part of the hash:\n merge :group {\n with @5 :Text;\n head @6 :Data;\n }\n}\n\nstruct DirEntry $Go.doc(\"A single directory entry\") {\n name @0 :Text;\n hash @1 :Data;\n}\n\nstruct Directory $Go.doc(\"Directory contains one or more directories or files\") {\n size @0 :UInt64;\n cachedSize @1 :Int64;\n parent @2 :Text;\n children @3 :List(DirEntry);\n contents @4 :List(DirEntry);\n}\n\nstruct File $Go.doc(\"A leaf node in the MDAG\") {\n size @0 :UInt64;\n cachedSize @1 :Int64;\n parent @2 :Text;\n key @3 :Data;\n\n # This indicates that the stream associated with this\n # file is not encoded by brig, but raw. We should not\n # attempt to decode it.\n isRaw @4 :Bool;\n}\n\nstruct Ghost $Go.doc(\"Ghost indicates that a certain node was at this path once\") {\n ghostInode @0 :UInt64;\n ghostPath @1 :Text;\n\n union {\n commit @2 :Commit;\n directory @3 :Directory;\n file @4 :File;\n }\n}\n\nstruct Node $Go.doc(\"Node is a node in the merkle dag of brig\") {\n name @0 :Text;\n treeHash @1 :Data;\n modTime @2 :Text; # Time as ISO8601\n inode @3 :UInt64;\n contentHash @4 :Data;\n user @5 :Text;\n\n union {\n commit @6 :Commit;\n directory @7 :Directory;\n file @8 :File;\n ghost @9 :Ghost;\n }\n\n backendHash @10 :Data;\n}\n" }, { "path": "catfs/nodes/capnp/nodes.capnp.go", "content": "// Code generated by capnpc-go. DO NOT EDIT.\n\npackage capnp\n\nimport (\n\tstrconv \"strconv\"\n\tcapnp \"zombiezen.com/go/capnproto2\"\n\ttext \"zombiezen.com/go/capnproto2/encoding/text\"\n\tschemas \"zombiezen.com/go/capnproto2/schemas\"\n)\n\n// Commit is a set of changes to nodes\ntype Commit struct{ capnp.Struct }\ntype Commit_merge Commit\n\n// Commit_TypeID is the unique identifier for the type Commit.\nconst Commit_TypeID = 0x8da013c66e545daf\n\nfunc NewCommit(s *capnp.Segment) (Commit, error) {\n\tst, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 8, PointerCount: 6})\n\treturn Commit{st}, err\n}\n\nfunc NewRootCommit(s *capnp.Segment) (Commit, error) {\n\tst, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 8, PointerCount: 6})\n\treturn Commit{st}, err\n}\n\nfunc ReadRootCommit(msg *capnp.Message) (Commit, error) {\n\troot, err := msg.RootPtr()\n\treturn Commit{root.Struct()}, err\n}\n\nfunc (s Commit) String() string {\n\tstr, _ := text.Marshal(0x8da013c66e545daf, s.Struct)\n\treturn str\n}\n\nfunc (s Commit) Message() (string, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.Text(), err\n}\n\nfunc (s Commit) HasMessage() bool {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Commit) MessageBytes() ([]byte, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.TextBytes(), err\n}\n\nfunc (s Commit) SetMessage(v string) error {\n\treturn s.Struct.SetText(0, v)\n}\n\nfunc (s Commit) Author() (string, error) {\n\tp, err := s.Struct.Ptr(1)\n\treturn p.Text(), err\n}\n\nfunc (s Commit) HasAuthor() bool {\n\tp, err := s.Struct.Ptr(1)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Commit) AuthorBytes() ([]byte, error) {\n\tp, err := s.Struct.Ptr(1)\n\treturn p.TextBytes(), err\n}\n\nfunc (s Commit) SetAuthor(v string) error {\n\treturn s.Struct.SetText(1, v)\n}\n\nfunc (s Commit) Parent() ([]byte, error) {\n\tp, err := s.Struct.Ptr(2)\n\treturn []byte(p.Data()), err\n}\n\nfunc (s Commit) HasParent() bool {\n\tp, err := s.Struct.Ptr(2)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Commit) SetParent(v []byte) error {\n\treturn s.Struct.SetData(2, v)\n}\n\nfunc (s Commit) Root() ([]byte, error) {\n\tp, err := s.Struct.Ptr(3)\n\treturn []byte(p.Data()), err\n}\n\nfunc (s Commit) HasRoot() bool {\n\tp, err := s.Struct.Ptr(3)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Commit) SetRoot(v []byte) error {\n\treturn s.Struct.SetData(3, v)\n}\n\nfunc (s Commit) Index() int64 {\n\treturn int64(s.Struct.Uint64(0))\n}\n\nfunc (s Commit) SetIndex(v int64) {\n\ts.Struct.SetUint64(0, uint64(v))\n}\n\nfunc (s Commit) Merge() Commit_merge { return Commit_merge(s) }\n\nfunc (s Commit_merge) With() (string, error) {\n\tp, err := s.Struct.Ptr(4)\n\treturn p.Text(), err\n}\n\nfunc (s Commit_merge) HasWith() bool {\n\tp, err := s.Struct.Ptr(4)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Commit_merge) WithBytes() ([]byte, error) {\n\tp, err := s.Struct.Ptr(4)\n\treturn p.TextBytes(), err\n}\n\nfunc (s Commit_merge) SetWith(v string) error {\n\treturn s.Struct.SetText(4, v)\n}\n\nfunc (s Commit_merge) Head() ([]byte, error) {\n\tp, err := s.Struct.Ptr(5)\n\treturn []byte(p.Data()), err\n}\n\nfunc (s Commit_merge) HasHead() bool {\n\tp, err := s.Struct.Ptr(5)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Commit_merge) SetHead(v []byte) error {\n\treturn s.Struct.SetData(5, v)\n}\n\n// Commit_List is a list of Commit.\ntype Commit_List struct{ capnp.List }\n\n// NewCommit creates a new list of Commit.\nfunc NewCommit_List(s *capnp.Segment, sz int32) (Commit_List, error) {\n\tl, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 8, PointerCount: 6}, sz)\n\treturn Commit_List{l}, err\n}\n\nfunc (s Commit_List) At(i int) Commit { return Commit{s.List.Struct(i)} }\n\nfunc (s Commit_List) Set(i int, v Commit) error { return s.List.SetStruct(i, v.Struct) }\n\nfunc (s Commit_List) String() string {\n\tstr, _ := text.MarshalList(0x8da013c66e545daf, s.List)\n\treturn str\n}\n\n// Commit_Promise is a wrapper for a Commit promised by a client call.\ntype Commit_Promise struct{ *capnp.Pipeline }\n\nfunc (p Commit_Promise) Struct() (Commit, error) {\n\ts, err := p.Pipeline.Struct()\n\treturn Commit{s}, err\n}\n\nfunc (p Commit_Promise) Merge() Commit_merge_Promise { return Commit_merge_Promise{p.Pipeline} }\n\n// Commit_merge_Promise is a wrapper for a Commit_merge promised by a client call.\ntype Commit_merge_Promise struct{ *capnp.Pipeline }\n\nfunc (p Commit_merge_Promise) Struct() (Commit_merge, error) {\n\ts, err := p.Pipeline.Struct()\n\treturn Commit_merge{s}, err\n}\n\n// A single directory entry\ntype DirEntry struct{ capnp.Struct }\n\n// DirEntry_TypeID is the unique identifier for the type DirEntry.\nconst DirEntry_TypeID = 0x8b15ee76774b1f9d\n\nfunc NewDirEntry(s *capnp.Segment) (DirEntry, error) {\n\tst, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 2})\n\treturn DirEntry{st}, err\n}\n\nfunc NewRootDirEntry(s *capnp.Segment) (DirEntry, error) {\n\tst, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 2})\n\treturn DirEntry{st}, err\n}\n\nfunc ReadRootDirEntry(msg *capnp.Message) (DirEntry, error) {\n\troot, err := msg.RootPtr()\n\treturn DirEntry{root.Struct()}, err\n}\n\nfunc (s DirEntry) String() string {\n\tstr, _ := text.Marshal(0x8b15ee76774b1f9d, s.Struct)\n\treturn str\n}\n\nfunc (s DirEntry) Name() (string, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.Text(), err\n}\n\nfunc (s DirEntry) HasName() bool {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s DirEntry) NameBytes() ([]byte, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.TextBytes(), err\n}\n\nfunc (s DirEntry) SetName(v string) error {\n\treturn s.Struct.SetText(0, v)\n}\n\nfunc (s DirEntry) Hash() ([]byte, error) {\n\tp, err := s.Struct.Ptr(1)\n\treturn []byte(p.Data()), err\n}\n\nfunc (s DirEntry) HasHash() bool {\n\tp, err := s.Struct.Ptr(1)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s DirEntry) SetHash(v []byte) error {\n\treturn s.Struct.SetData(1, v)\n}\n\n// DirEntry_List is a list of DirEntry.\ntype DirEntry_List struct{ capnp.List }\n\n// NewDirEntry creates a new list of DirEntry.\nfunc NewDirEntry_List(s *capnp.Segment, sz int32) (DirEntry_List, error) {\n\tl, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 0, PointerCount: 2}, sz)\n\treturn DirEntry_List{l}, err\n}\n\nfunc (s DirEntry_List) At(i int) DirEntry { return DirEntry{s.List.Struct(i)} }\n\nfunc (s DirEntry_List) Set(i int, v DirEntry) error { return s.List.SetStruct(i, v.Struct) }\n\nfunc (s DirEntry_List) String() string {\n\tstr, _ := text.MarshalList(0x8b15ee76774b1f9d, s.List)\n\treturn str\n}\n\n// DirEntry_Promise is a wrapper for a DirEntry promised by a client call.\ntype DirEntry_Promise struct{ *capnp.Pipeline }\n\nfunc (p DirEntry_Promise) Struct() (DirEntry, error) {\n\ts, err := p.Pipeline.Struct()\n\treturn DirEntry{s}, err\n}\n\n// Directory contains one or more directories or files\ntype Directory struct{ capnp.Struct }\n\n// Directory_TypeID is the unique identifier for the type Directory.\nconst Directory_TypeID = 0xe24c59306c829c01\n\nfunc NewDirectory(s *capnp.Segment) (Directory, error) {\n\tst, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 16, PointerCount: 3})\n\treturn Directory{st}, err\n}\n\nfunc NewRootDirectory(s *capnp.Segment) (Directory, error) {\n\tst, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 16, PointerCount: 3})\n\treturn Directory{st}, err\n}\n\nfunc ReadRootDirectory(msg *capnp.Message) (Directory, error) {\n\troot, err := msg.RootPtr()\n\treturn Directory{root.Struct()}, err\n}\n\nfunc (s Directory) String() string {\n\tstr, _ := text.Marshal(0xe24c59306c829c01, s.Struct)\n\treturn str\n}\n\nfunc (s Directory) Size() uint64 {\n\treturn s.Struct.Uint64(0)\n}\n\nfunc (s Directory) SetSize(v uint64) {\n\ts.Struct.SetUint64(0, v)\n}\n\nfunc (s Directory) CachedSize() int64 {\n\treturn int64(s.Struct.Uint64(8))\n}\n\nfunc (s Directory) SetCachedSize(v int64) {\n\ts.Struct.SetUint64(8, uint64(v))\n}\n\nfunc (s Directory) Parent() (string, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.Text(), err\n}\n\nfunc (s Directory) HasParent() bool {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Directory) ParentBytes() ([]byte, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.TextBytes(), err\n}\n\nfunc (s Directory) SetParent(v string) error {\n\treturn s.Struct.SetText(0, v)\n}\n\nfunc (s Directory) Children() (DirEntry_List, error) {\n\tp, err := s.Struct.Ptr(1)\n\treturn DirEntry_List{List: p.List()}, err\n}\n\nfunc (s Directory) HasChildren() bool {\n\tp, err := s.Struct.Ptr(1)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Directory) SetChildren(v DirEntry_List) error {\n\treturn s.Struct.SetPtr(1, v.List.ToPtr())\n}\n\n// NewChildren sets the children field to a newly\n// allocated DirEntry_List, preferring placement in s's segment.\nfunc (s Directory) NewChildren(n int32) (DirEntry_List, error) {\n\tl, err := NewDirEntry_List(s.Struct.Segment(), n)\n\tif err != nil {\n\t\treturn DirEntry_List{}, err\n\t}\n\terr = s.Struct.SetPtr(1, l.List.ToPtr())\n\treturn l, err\n}\n\nfunc (s Directory) Contents() (DirEntry_List, error) {\n\tp, err := s.Struct.Ptr(2)\n\treturn DirEntry_List{List: p.List()}, err\n}\n\nfunc (s Directory) HasContents() bool {\n\tp, err := s.Struct.Ptr(2)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Directory) SetContents(v DirEntry_List) error {\n\treturn s.Struct.SetPtr(2, v.List.ToPtr())\n}\n\n// NewContents sets the contents field to a newly\n// allocated DirEntry_List, preferring placement in s's segment.\nfunc (s Directory) NewContents(n int32) (DirEntry_List, error) {\n\tl, err := NewDirEntry_List(s.Struct.Segment(), n)\n\tif err != nil {\n\t\treturn DirEntry_List{}, err\n\t}\n\terr = s.Struct.SetPtr(2, l.List.ToPtr())\n\treturn l, err\n}\n\n// Directory_List is a list of Directory.\ntype Directory_List struct{ capnp.List }\n\n// NewDirectory creates a new list of Directory.\nfunc NewDirectory_List(s *capnp.Segment, sz int32) (Directory_List, error) {\n\tl, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 16, PointerCount: 3}, sz)\n\treturn Directory_List{l}, err\n}\n\nfunc (s Directory_List) At(i int) Directory { return Directory{s.List.Struct(i)} }\n\nfunc (s Directory_List) Set(i int, v Directory) error { return s.List.SetStruct(i, v.Struct) }\n\nfunc (s Directory_List) String() string {\n\tstr, _ := text.MarshalList(0xe24c59306c829c01, s.List)\n\treturn str\n}\n\n// Directory_Promise is a wrapper for a Directory promised by a client call.\ntype Directory_Promise struct{ *capnp.Pipeline }\n\nfunc (p Directory_Promise) Struct() (Directory, error) {\n\ts, err := p.Pipeline.Struct()\n\treturn Directory{s}, err\n}\n\n// A leaf node in the MDAG\ntype File struct{ capnp.Struct }\n\n// File_TypeID is the unique identifier for the type File.\nconst File_TypeID = 0x8ea7393d37893155\n\nfunc NewFile(s *capnp.Segment) (File, error) {\n\tst, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 24, PointerCount: 2})\n\treturn File{st}, err\n}\n\nfunc NewRootFile(s *capnp.Segment) (File, error) {\n\tst, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 24, PointerCount: 2})\n\treturn File{st}, err\n}\n\nfunc ReadRootFile(msg *capnp.Message) (File, error) {\n\troot, err := msg.RootPtr()\n\treturn File{root.Struct()}, err\n}\n\nfunc (s File) String() string {\n\tstr, _ := text.Marshal(0x8ea7393d37893155, s.Struct)\n\treturn str\n}\n\nfunc (s File) Size() uint64 {\n\treturn s.Struct.Uint64(0)\n}\n\nfunc (s File) SetSize(v uint64) {\n\ts.Struct.SetUint64(0, v)\n}\n\nfunc (s File) CachedSize() int64 {\n\treturn int64(s.Struct.Uint64(8))\n}\n\nfunc (s File) SetCachedSize(v int64) {\n\ts.Struct.SetUint64(8, uint64(v))\n}\n\nfunc (s File) Parent() (string, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.Text(), err\n}\n\nfunc (s File) HasParent() bool {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s File) ParentBytes() ([]byte, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.TextBytes(), err\n}\n\nfunc (s File) SetParent(v string) error {\n\treturn s.Struct.SetText(0, v)\n}\n\nfunc (s File) Key() ([]byte, error) {\n\tp, err := s.Struct.Ptr(1)\n\treturn []byte(p.Data()), err\n}\n\nfunc (s File) HasKey() bool {\n\tp, err := s.Struct.Ptr(1)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s File) SetKey(v []byte) error {\n\treturn s.Struct.SetData(1, v)\n}\n\nfunc (s File) IsRaw() bool {\n\treturn s.Struct.Bit(128)\n}\n\nfunc (s File) SetIsRaw(v bool) {\n\ts.Struct.SetBit(128, v)\n}\n\n// File_List is a list of File.\ntype File_List struct{ capnp.List }\n\n// NewFile creates a new list of File.\nfunc NewFile_List(s *capnp.Segment, sz int32) (File_List, error) {\n\tl, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 24, PointerCount: 2}, sz)\n\treturn File_List{l}, err\n}\n\nfunc (s File_List) At(i int) File { return File{s.List.Struct(i)} }\n\nfunc (s File_List) Set(i int, v File) error { return s.List.SetStruct(i, v.Struct) }\n\nfunc (s File_List) String() string {\n\tstr, _ := text.MarshalList(0x8ea7393d37893155, s.List)\n\treturn str\n}\n\n// File_Promise is a wrapper for a File promised by a client call.\ntype File_Promise struct{ *capnp.Pipeline }\n\nfunc (p File_Promise) Struct() (File, error) {\n\ts, err := p.Pipeline.Struct()\n\treturn File{s}, err\n}\n\n// Ghost indicates that a certain node was at this path once\ntype Ghost struct{ capnp.Struct }\ntype Ghost_Which uint16\n\nconst (\n\tGhost_Which_commit Ghost_Which = 0\n\tGhost_Which_directory Ghost_Which = 1\n\tGhost_Which_file Ghost_Which = 2\n)\n\nfunc (w Ghost_Which) String() string {\n\tconst s = \"commitdirectoryfile\"\n\tswitch w {\n\tcase Ghost_Which_commit:\n\t\treturn s[0:6]\n\tcase Ghost_Which_directory:\n\t\treturn s[6:15]\n\tcase Ghost_Which_file:\n\t\treturn s[15:19]\n\n\t}\n\treturn \"Ghost_Which(\" + strconv.FormatUint(uint64(w), 10) + \")\"\n}\n\n// Ghost_TypeID is the unique identifier for the type Ghost.\nconst Ghost_TypeID = 0x80c828d7e89c12ea\n\nfunc NewGhost(s *capnp.Segment) (Ghost, error) {\n\tst, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 16, PointerCount: 2})\n\treturn Ghost{st}, err\n}\n\nfunc NewRootGhost(s *capnp.Segment) (Ghost, error) {\n\tst, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 16, PointerCount: 2})\n\treturn Ghost{st}, err\n}\n\nfunc ReadRootGhost(msg *capnp.Message) (Ghost, error) {\n\troot, err := msg.RootPtr()\n\treturn Ghost{root.Struct()}, err\n}\n\nfunc (s Ghost) String() string {\n\tstr, _ := text.Marshal(0x80c828d7e89c12ea, s.Struct)\n\treturn str\n}\n\nfunc (s Ghost) Which() Ghost_Which {\n\treturn Ghost_Which(s.Struct.Uint16(8))\n}\nfunc (s Ghost) GhostInode() uint64 {\n\treturn s.Struct.Uint64(0)\n}\n\nfunc (s Ghost) SetGhostInode(v uint64) {\n\ts.Struct.SetUint64(0, v)\n}\n\nfunc (s Ghost) GhostPath() (string, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.Text(), err\n}\n\nfunc (s Ghost) HasGhostPath() bool {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Ghost) GhostPathBytes() ([]byte, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.TextBytes(), err\n}\n\nfunc (s Ghost) SetGhostPath(v string) error {\n\treturn s.Struct.SetText(0, v)\n}\n\nfunc (s Ghost) Commit() (Commit, error) {\n\tif s.Struct.Uint16(8) != 0 {\n\t\tpanic(\"Which() != commit\")\n\t}\n\tp, err := s.Struct.Ptr(1)\n\treturn Commit{Struct: p.Struct()}, err\n}\n\nfunc (s Ghost) HasCommit() bool {\n\tif s.Struct.Uint16(8) != 0 {\n\t\treturn false\n\t}\n\tp, err := s.Struct.Ptr(1)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Ghost) SetCommit(v Commit) error {\n\ts.Struct.SetUint16(8, 0)\n\treturn s.Struct.SetPtr(1, v.Struct.ToPtr())\n}\n\n// NewCommit sets the commit field to a newly\n// allocated Commit struct, preferring placement in s's segment.\nfunc (s Ghost) NewCommit() (Commit, error) {\n\ts.Struct.SetUint16(8, 0)\n\tss, err := NewCommit(s.Struct.Segment())\n\tif err != nil {\n\t\treturn Commit{}, err\n\t}\n\terr = s.Struct.SetPtr(1, ss.Struct.ToPtr())\n\treturn ss, err\n}\n\nfunc (s Ghost) Directory() (Directory, error) {\n\tif s.Struct.Uint16(8) != 1 {\n\t\tpanic(\"Which() != directory\")\n\t}\n\tp, err := s.Struct.Ptr(1)\n\treturn Directory{Struct: p.Struct()}, err\n}\n\nfunc (s Ghost) HasDirectory() bool {\n\tif s.Struct.Uint16(8) != 1 {\n\t\treturn false\n\t}\n\tp, err := s.Struct.Ptr(1)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Ghost) SetDirectory(v Directory) error {\n\ts.Struct.SetUint16(8, 1)\n\treturn s.Struct.SetPtr(1, v.Struct.ToPtr())\n}\n\n// NewDirectory sets the directory field to a newly\n// allocated Directory struct, preferring placement in s's segment.\nfunc (s Ghost) NewDirectory() (Directory, error) {\n\ts.Struct.SetUint16(8, 1)\n\tss, err := NewDirectory(s.Struct.Segment())\n\tif err != nil {\n\t\treturn Directory{}, err\n\t}\n\terr = s.Struct.SetPtr(1, ss.Struct.ToPtr())\n\treturn ss, err\n}\n\nfunc (s Ghost) File() (File, error) {\n\tif s.Struct.Uint16(8) != 2 {\n\t\tpanic(\"Which() != file\")\n\t}\n\tp, err := s.Struct.Ptr(1)\n\treturn File{Struct: p.Struct()}, err\n}\n\nfunc (s Ghost) HasFile() bool {\n\tif s.Struct.Uint16(8) != 2 {\n\t\treturn false\n\t}\n\tp, err := s.Struct.Ptr(1)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Ghost) SetFile(v File) error {\n\ts.Struct.SetUint16(8, 2)\n\treturn s.Struct.SetPtr(1, v.Struct.ToPtr())\n}\n\n// NewFile sets the file field to a newly\n// allocated File struct, preferring placement in s's segment.\nfunc (s Ghost) NewFile() (File, error) {\n\ts.Struct.SetUint16(8, 2)\n\tss, err := NewFile(s.Struct.Segment())\n\tif err != nil {\n\t\treturn File{}, err\n\t}\n\terr = s.Struct.SetPtr(1, ss.Struct.ToPtr())\n\treturn ss, err\n}\n\n// Ghost_List is a list of Ghost.\ntype Ghost_List struct{ capnp.List }\n\n// NewGhost creates a new list of Ghost.\nfunc NewGhost_List(s *capnp.Segment, sz int32) (Ghost_List, error) {\n\tl, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 16, PointerCount: 2}, sz)\n\treturn Ghost_List{l}, err\n}\n\nfunc (s Ghost_List) At(i int) Ghost { return Ghost{s.List.Struct(i)} }\n\nfunc (s Ghost_List) Set(i int, v Ghost) error { return s.List.SetStruct(i, v.Struct) }\n\nfunc (s Ghost_List) String() string {\n\tstr, _ := text.MarshalList(0x80c828d7e89c12ea, s.List)\n\treturn str\n}\n\n// Ghost_Promise is a wrapper for a Ghost promised by a client call.\ntype Ghost_Promise struct{ *capnp.Pipeline }\n\nfunc (p Ghost_Promise) Struct() (Ghost, error) {\n\ts, err := p.Pipeline.Struct()\n\treturn Ghost{s}, err\n}\n\nfunc (p Ghost_Promise) Commit() Commit_Promise {\n\treturn Commit_Promise{Pipeline: p.Pipeline.GetPipeline(1)}\n}\n\nfunc (p Ghost_Promise) Directory() Directory_Promise {\n\treturn Directory_Promise{Pipeline: p.Pipeline.GetPipeline(1)}\n}\n\nfunc (p Ghost_Promise) File() File_Promise {\n\treturn File_Promise{Pipeline: p.Pipeline.GetPipeline(1)}\n}\n\n// Node is a node in the merkle dag of brig\ntype Node struct{ capnp.Struct }\ntype Node_Which uint16\n\nconst (\n\tNode_Which_commit Node_Which = 0\n\tNode_Which_directory Node_Which = 1\n\tNode_Which_file Node_Which = 2\n\tNode_Which_ghost Node_Which = 3\n)\n\nfunc (w Node_Which) String() string {\n\tconst s = \"commitdirectoryfileghost\"\n\tswitch w {\n\tcase Node_Which_commit:\n\t\treturn s[0:6]\n\tcase Node_Which_directory:\n\t\treturn s[6:15]\n\tcase Node_Which_file:\n\t\treturn s[15:19]\n\tcase Node_Which_ghost:\n\t\treturn s[19:24]\n\n\t}\n\treturn \"Node_Which(\" + strconv.FormatUint(uint64(w), 10) + \")\"\n}\n\n// Node_TypeID is the unique identifier for the type Node.\nconst Node_TypeID = 0xa629eb7f7066fae3\n\nfunc NewNode(s *capnp.Segment) (Node, error) {\n\tst, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 16, PointerCount: 7})\n\treturn Node{st}, err\n}\n\nfunc NewRootNode(s *capnp.Segment) (Node, error) {\n\tst, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 16, PointerCount: 7})\n\treturn Node{st}, err\n}\n\nfunc ReadRootNode(msg *capnp.Message) (Node, error) {\n\troot, err := msg.RootPtr()\n\treturn Node{root.Struct()}, err\n}\n\nfunc (s Node) String() string {\n\tstr, _ := text.Marshal(0xa629eb7f7066fae3, s.Struct)\n\treturn str\n}\n\nfunc (s Node) Which() Node_Which {\n\treturn Node_Which(s.Struct.Uint16(8))\n}\nfunc (s Node) Name() (string, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.Text(), err\n}\n\nfunc (s Node) HasName() bool {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Node) NameBytes() ([]byte, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.TextBytes(), err\n}\n\nfunc (s Node) SetName(v string) error {\n\treturn s.Struct.SetText(0, v)\n}\n\nfunc (s Node) TreeHash() ([]byte, error) {\n\tp, err := s.Struct.Ptr(1)\n\treturn []byte(p.Data()), err\n}\n\nfunc (s Node) HasTreeHash() bool {\n\tp, err := s.Struct.Ptr(1)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Node) SetTreeHash(v []byte) error {\n\treturn s.Struct.SetData(1, v)\n}\n\nfunc (s Node) ModTime() (string, error) {\n\tp, err := s.Struct.Ptr(2)\n\treturn p.Text(), err\n}\n\nfunc (s Node) HasModTime() bool {\n\tp, err := s.Struct.Ptr(2)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Node) ModTimeBytes() ([]byte, error) {\n\tp, err := s.Struct.Ptr(2)\n\treturn p.TextBytes(), err\n}\n\nfunc (s Node) SetModTime(v string) error {\n\treturn s.Struct.SetText(2, v)\n}\n\nfunc (s Node) Inode() uint64 {\n\treturn s.Struct.Uint64(0)\n}\n\nfunc (s Node) SetInode(v uint64) {\n\ts.Struct.SetUint64(0, v)\n}\n\nfunc (s Node) ContentHash() ([]byte, error) {\n\tp, err := s.Struct.Ptr(3)\n\treturn []byte(p.Data()), err\n}\n\nfunc (s Node) HasContentHash() bool {\n\tp, err := s.Struct.Ptr(3)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Node) SetContentHash(v []byte) error {\n\treturn s.Struct.SetData(3, v)\n}\n\nfunc (s Node) User() (string, error) {\n\tp, err := s.Struct.Ptr(4)\n\treturn p.Text(), err\n}\n\nfunc (s Node) HasUser() bool {\n\tp, err := s.Struct.Ptr(4)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Node) UserBytes() ([]byte, error) {\n\tp, err := s.Struct.Ptr(4)\n\treturn p.TextBytes(), err\n}\n\nfunc (s Node) SetUser(v string) error {\n\treturn s.Struct.SetText(4, v)\n}\n\nfunc (s Node) Commit() (Commit, error) {\n\tif s.Struct.Uint16(8) != 0 {\n\t\tpanic(\"Which() != commit\")\n\t}\n\tp, err := s.Struct.Ptr(5)\n\treturn Commit{Struct: p.Struct()}, err\n}\n\nfunc (s Node) HasCommit() bool {\n\tif s.Struct.Uint16(8) != 0 {\n\t\treturn false\n\t}\n\tp, err := s.Struct.Ptr(5)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Node) SetCommit(v Commit) error {\n\ts.Struct.SetUint16(8, 0)\n\treturn s.Struct.SetPtr(5, v.Struct.ToPtr())\n}\n\n// NewCommit sets the commit field to a newly\n// allocated Commit struct, preferring placement in s's segment.\nfunc (s Node) NewCommit() (Commit, error) {\n\ts.Struct.SetUint16(8, 0)\n\tss, err := NewCommit(s.Struct.Segment())\n\tif err != nil {\n\t\treturn Commit{}, err\n\t}\n\terr = s.Struct.SetPtr(5, ss.Struct.ToPtr())\n\treturn ss, err\n}\n\nfunc (s Node) Directory() (Directory, error) {\n\tif s.Struct.Uint16(8) != 1 {\n\t\tpanic(\"Which() != directory\")\n\t}\n\tp, err := s.Struct.Ptr(5)\n\treturn Directory{Struct: p.Struct()}, err\n}\n\nfunc (s Node) HasDirectory() bool {\n\tif s.Struct.Uint16(8) != 1 {\n\t\treturn false\n\t}\n\tp, err := s.Struct.Ptr(5)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Node) SetDirectory(v Directory) error {\n\ts.Struct.SetUint16(8, 1)\n\treturn s.Struct.SetPtr(5, v.Struct.ToPtr())\n}\n\n// NewDirectory sets the directory field to a newly\n// allocated Directory struct, preferring placement in s's segment.\nfunc (s Node) NewDirectory() (Directory, error) {\n\ts.Struct.SetUint16(8, 1)\n\tss, err := NewDirectory(s.Struct.Segment())\n\tif err != nil {\n\t\treturn Directory{}, err\n\t}\n\terr = s.Struct.SetPtr(5, ss.Struct.ToPtr())\n\treturn ss, err\n}\n\nfunc (s Node) File() (File, error) {\n\tif s.Struct.Uint16(8) != 2 {\n\t\tpanic(\"Which() != file\")\n\t}\n\tp, err := s.Struct.Ptr(5)\n\treturn File{Struct: p.Struct()}, err\n}\n\nfunc (s Node) HasFile() bool {\n\tif s.Struct.Uint16(8) != 2 {\n\t\treturn false\n\t}\n\tp, err := s.Struct.Ptr(5)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Node) SetFile(v File) error {\n\ts.Struct.SetUint16(8, 2)\n\treturn s.Struct.SetPtr(5, v.Struct.ToPtr())\n}\n\n// NewFile sets the file field to a newly\n// allocated File struct, preferring placement in s's segment.\nfunc (s Node) NewFile() (File, error) {\n\ts.Struct.SetUint16(8, 2)\n\tss, err := NewFile(s.Struct.Segment())\n\tif err != nil {\n\t\treturn File{}, err\n\t}\n\terr = s.Struct.SetPtr(5, ss.Struct.ToPtr())\n\treturn ss, err\n}\n\nfunc (s Node) Ghost() (Ghost, error) {\n\tif s.Struct.Uint16(8) != 3 {\n\t\tpanic(\"Which() != ghost\")\n\t}\n\tp, err := s.Struct.Ptr(5)\n\treturn Ghost{Struct: p.Struct()}, err\n}\n\nfunc (s Node) HasGhost() bool {\n\tif s.Struct.Uint16(8) != 3 {\n\t\treturn false\n\t}\n\tp, err := s.Struct.Ptr(5)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Node) SetGhost(v Ghost) error {\n\ts.Struct.SetUint16(8, 3)\n\treturn s.Struct.SetPtr(5, v.Struct.ToPtr())\n}\n\n// NewGhost sets the ghost field to a newly\n// allocated Ghost struct, preferring placement in s's segment.\nfunc (s Node) NewGhost() (Ghost, error) {\n\ts.Struct.SetUint16(8, 3)\n\tss, err := NewGhost(s.Struct.Segment())\n\tif err != nil {\n\t\treturn Ghost{}, err\n\t}\n\terr = s.Struct.SetPtr(5, ss.Struct.ToPtr())\n\treturn ss, err\n}\n\nfunc (s Node) BackendHash() ([]byte, error) {\n\tp, err := s.Struct.Ptr(6)\n\treturn []byte(p.Data()), err\n}\n\nfunc (s Node) HasBackendHash() bool {\n\tp, err := s.Struct.Ptr(6)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Node) SetBackendHash(v []byte) error {\n\treturn s.Struct.SetData(6, v)\n}\n\n// Node_List is a list of Node.\ntype Node_List struct{ capnp.List }\n\n// NewNode creates a new list of Node.\nfunc NewNode_List(s *capnp.Segment, sz int32) (Node_List, error) {\n\tl, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 16, PointerCount: 7}, sz)\n\treturn Node_List{l}, err\n}\n\nfunc (s Node_List) At(i int) Node { return Node{s.List.Struct(i)} }\n\nfunc (s Node_List) Set(i int, v Node) error { return s.List.SetStruct(i, v.Struct) }\n\nfunc (s Node_List) String() string {\n\tstr, _ := text.MarshalList(0xa629eb7f7066fae3, s.List)\n\treturn str\n}\n\n// Node_Promise is a wrapper for a Node promised by a client call.\ntype Node_Promise struct{ *capnp.Pipeline }\n\nfunc (p Node_Promise) Struct() (Node, error) {\n\ts, err := p.Pipeline.Struct()\n\treturn Node{s}, err\n}\n\nfunc (p Node_Promise) Commit() Commit_Promise {\n\treturn Commit_Promise{Pipeline: p.Pipeline.GetPipeline(5)}\n}\n\nfunc (p Node_Promise) Directory() Directory_Promise {\n\treturn Directory_Promise{Pipeline: p.Pipeline.GetPipeline(5)}\n}\n\nfunc (p Node_Promise) File() File_Promise {\n\treturn File_Promise{Pipeline: p.Pipeline.GetPipeline(5)}\n}\n\nfunc (p Node_Promise) Ghost() Ghost_Promise {\n\treturn Ghost_Promise{Pipeline: p.Pipeline.GetPipeline(5)}\n}\n\nconst schema_9195d073cb5c5953 = \"x\\xda\\xb4\\x96\\xefk\\x14\\xd7\\x1a\\xc7\\x9f\\xef9\\xb3;Y\" +\n\t\"\\xd1\\xbb\\xd9;\\x11D\\\\\\xf7 \\x0aQ\\x12M\\\\/W\" +\n\t\"\\x83\\x17\\x8d\\xc6k\\xcc\\xcd\\x95\\x1cW\\xb9*za\\xdc=\" +\n\t\"\\xd9\\x1d\\xdc\\x9d\\x093cc\\xa4E[\\x14\\xb45E\\xa9\" +\n\t\"\\x82BBm\\x89\\xfd\\x01-m\\xff\\x80\\xd2\\x1f`)\\xed\" +\n\t\"\\x9b\\xd2\\x17-\\xed\\xcb\\xb6/j\\x0b}]\\xda\\xea\\x94\\xb3\" +\n\t\"\\xbf&\\x86\\xb4Ji\\xdfe\\x9e\\xe7\\xfc\\xf8\\x9e\\xcf\\xf3<\" +\n\t\"\\xdfl\\xdf\\xcfl'\\xebOx\\x8cH\\xaeJ$\\xa3\\xef\" +\n\t\"\\xfe>\\xf3\\xed\\xe7\\xdd\\x1f\\x9e#\\xb9\\x1a,*\\x1c9\\xf6\" +\n\t\"q\\xf0\\xc9\\xf5\\xab\\xb4\\x87\\x99\\x1cF>\\x8b5\\xb0za\" +\n\t\"Z\\xbd\\xc8\\xe5\\x1d\\xe4@\\x88fs\\xff\\x99|\\xec\\x87\\xe5\" +\n\t\"\\xcfPf5\\xe2\\x0d\\x09f\\x12\\xe5/\\xb0\\x01X\\xd7\\x99\" +\n\t\"i]g9\\xeb=6I\\x88\\xde8~\\xd0\\xfd\\xc0\\xba\" +\n\t\"5\\xad/\\x98\\xbf>\\xa9\\xd7g\\xf9\\x06X\\xbd\\xdc\\xb4z\" +\n\t\"y.\\x7f\\x9c\\xffO\\x9f\\x7f\\xa8\\xff\\xd2?\\xff\\xb5\\xed\\xe5\" +\n\t\"g\\xf5\\x06\\xbe\\xf0\\x82ic%\\xacY\\xc3\\xb4f\\x8d\\\\\" +\n\t\"\\xfe\\x8eQ\\x17\\xf4\\xf5O\\xe3\\x13g\\xbf_\\xff\\xd2\\xc2'\" +\n\t\"\\x98\\xa6\\x01#\\xffEb%\\xac\\xbb\\x09\\xd3\\xba\\x9b\\xc8\\xe5\" +\n\t\"\\xb3I\\x8f\\x11\\xa2\\xb9oF\\xbfL\\xcf\\xfd\\xf8\\x0e\\xc9u\" +\n\t\"\\x98\\xa7py\\xd2\\x04Q\\xfev\\xc7Q\\x10\\xac\\xb7:\\xb4\" +\n\t\"|\\xcc\\x15V\" +\n\t\"<\\xbf\\xfd9a\\xfb\\xca\\x0d[d\\xd2\\xbe\\xe7\\xb5?r\" +\n\t\"\\x8e[R\\xa7\\x91 \\x86\\x04!WS~Y=\\x0c\\xdd\" +\n\t\"\\xbf\\x1d^U\\x8b\\x83[\\xd1,\\xf0\\xfb\\xd1\\xa0\\xa8*{\" +\n\t\"\\\\\\xb8LO\\x8d\\xe3\\x8a\\xb0\\xa2\\xc4\\x7f\\x87\\x06\\xf7\\xd2\\x83\" +\n\t\"\\xb3\\xa2\\x1f{\\x9aC\\x9e\\x9f\\xc7\\xeaI=@\\xe78\\xe4\" +\n\t\"e\\x064Q]\\x1a\\x88\\x07(\\xc3\\x9bs2\\xbd&\\x1e\" +\n\t\"\\x9f\\x8cq\\xae\\xc1\\xea\\xaafu\\x99C\\xde`H\\x07\\xce\" +\n\t\"\\x99x:\\x8av\\xb1\\xa2J\\x05\\x87\\xf8\\x19\\xd5zq\\x8b\" +\n\t\"M\\x13\\x95yRM\\xc5h\\x82\\x03\\xf6$@\\x0c\\x8f`\" +\n\t\"2\\xfbubq(k\\x9b\\xdd4\\x82h\\x7f\\x9dF \" +\n\t\"\\x0c\\xbba'M05\\xe5\\x9f\\xac*Q\\xb2\\xcb\\xba\\xbd\" +\n\t\"N\\xf8N\\x99 {Z\\x94\\xacu\\xd8@T\\x10\\xe0(\" +\n\t\"\\xf4 \\x06e\\xad\\xc7\\x08Q\\xa1[\\xc7\\xb7 \\xee+\\xab\" +\n\t\"\\x1f\\xbb\\x88\\x0a=:\\xbe\\x15\\x0cht\\x96\\xf5\\x0fl&\" +\n\t\"*\\xf4\\xe9\\xf0v\\xbd\\xdc\\xe0ub\\xd66\\x9c *l\" +\n\t\"\\xd5\\xf1!\\x1dO\\x18]H\\x10Y\\x83\\xf5k\\xb7\\xeb\\xf8\" +\n\t\"0\\x18\\xb2\\xc9(Jt!Id\\xed\\xc1\\x00Qa\\xa7\" +\n\t\"\\xce\\x8c\\xea\\x8cy_gL\\\"k\\x1f\\x0e\\x10\\x15\\x86u\" +\n\t\"\\xe6\\xa0\\xcet\\xdc\\xd3\\x99\\x0e\\\"K\\xd6O\\x1b\\xd5\\x99\\xc3\" +\n\t\":\\x93\\xfaEgRD\\xd6\\xa1\\xba\\xae1\\x9d9\\xa6\\xef\" +\n\t\"_\\x92\\xec\\xc2\\x12\\\"\\xebH]\\xd7a\\x1d/a\\xc1\\xac\" +\n\t\"G\\xa1\\xaf\\xd4\\xb0\\x1dT\\x88\\xa8U\\xb0\\xb35\\xaft\\xd0\" +\n\t\"\\x89\\xd7\\xe4\\x1c\\xcd8.\\xbf\\xe7\\x86\\xca\\x0d\\x87\\xc9\\x9cg\" +\n\t\"\\x13\\xe9S\\x81\\xf2\\xff\\x1a\\xaf\\xcc\\xd5\\xdd\\x18\\x9d\\xf1\\xff\\xfb\" +\n\t\"\\xe6a'\\xec\\xe2I\\xe5\\x96\\x1e\\x14\\xd2\\xee/\\xe3\\xb7\\xfc\" +\n\t\"JK\\xdbXS>/+m\\x91\\x9d\\x8d*-\\xf0\\xc8\" +\n\t\"F\\x81\\x1e\\xf4\\xc8I'\\xac\\xc4\\x1e\\xa9\\xec\\xd2\\xa3\\xde9\" +\n\t\"\\xd4\\xb2fZ\\xbc\\xb1\\xbb\\x9b\\x8d\\xfd\\\"\\xa2\\xd6\\xd2\\xc4\\x94\" +\n\t\"\\xd0\\x9cm\\xc7\\x0d\\x84\\xe7*\\xe1\\xf9\\xa2\\xe6\\xf9\\xaa\\xed\\xf2\" +\n\t\"\\x8e\\x0atl\\xdc1\\xabu\\xdb\\xfc\\xd3\\xac`\\xa49\\xf5\" +\n\t\"\\xdaJ\\x0d\\xd6\\xb0\\x82\\xdb#M\\xdb|\\xf7\\x0fXAT\" +\n\t\"\\xac8\\xd5\\x92\\xaf\\\\\\xdda\\x7f#\\x8cq\\xa03\\xfe%\" +\n\t\"F\\xd0\\xc1VS\\x05\\xbf\\xb7\\xe8\\xd7\\x00\\x00\\x00\\xff\\xff*\" +\n\t\"\\xb6Z]\"\n\nfunc init() {\n\tschemas.Register(schema_9195d073cb5c5953,\n\t\t0x80c828d7e89c12ea,\n\t\t0x8b15ee76774b1f9d,\n\t\t0x8da013c66e545daf,\n\t\t0x8ea7393d37893155,\n\t\t0xa629eb7f7066fae3,\n\t\t0xbff8a40fda4ce4a4,\n\t\t0xe24c59306c829c01)\n}\n" }, { "path": "catfs/nodes/commit.go", "content": "package nodes\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"path\"\n\t\"time\"\n\n\tcapnp_model \"github.com/sahib/brig/catfs/nodes/capnp\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\tcapnp \"zombiezen.com/go/capnproto2\"\n)\n\nconst (\n\t// AuthorOfStage is the Person that is displayed for the stage commit.\n\t// Currently this is just an empty hash Person that will be set later.\n\tAuthorOfStage = \"unknown\"\n)\n\n// Commit groups a set of changes\ntype Commit struct {\n\tBase\n\n\t// Commit message (might be auto-generated)\n\tmessage string\n\n\t// Author is the id of the committer.\n\tauthor string\n\n\t// root is the tree hash of the root directory\n\troot h.Hash\n\n\t// Parent hash (only nil for initial commit)\n\tparent h.Hash\n\n\t// Index of the commit (first is 0, second 1 and so on)\n\tindex int64\n\n\tmerge struct {\n\t\t// With indicates with which person we merged.\n\t\twith string\n\n\t\t// head is a reference to the commit we merged with on\n\t\t// the remote side.\n\t\thead h.Hash\n\t}\n}\n\n// NewEmptyCommit creates a new commit after the commit referenced by `parent`.\n// `parent` might be nil for the very first commit.\nfunc NewEmptyCommit(inode uint64, index int64) (*Commit, error) {\n\treturn &Commit{\n\t\tBase: Base{\n\t\t\tnodeType: NodeTypeCommit,\n\t\t\tinode: inode,\n\t\t\tmodTime: time.Now(),\n\t\t},\n\t\tindex: index,\n\t\tauthor: AuthorOfStage,\n\t}, nil\n}\n\n// ToCapnp will convert all commit internals to a capnp message.\nfunc (c *Commit) ToCapnp() (*capnp.Message, error) {\n\tmsg, seg, err := capnp.NewMessage(capnp.SingleSegment(nil))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapNd, err := capnp_model.NewRootNode(seg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn msg, c.ToCapnpNode(seg, capNd)\n}\n\n// ToCapnpNode converts this node to a serializable capnp proto node.\nfunc (c *Commit) ToCapnpNode(seg *capnp.Segment, capNd capnp_model.Node) error {\n\tif err := c.setBaseAttrsToNode(capNd); err != nil {\n\t\treturn err\n\t}\n\n\tcapCmt, err := c.setCommitAttrs(seg)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn capNd.SetCommit(*capCmt)\n}\n\nfunc (c *Commit) setCommitAttrs(seg *capnp.Segment) (*capnp_model.Commit, error) {\n\tcapCmt, err := capnp_model.NewCommit(seg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := capCmt.SetMessage(c.message); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := capCmt.SetRoot(c.root); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := capCmt.SetAuthor(c.author); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := capCmt.SetParent(c.parent); err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapCmt.SetIndex(c.index)\n\n\t// Store merge infos:\n\tcapmerge := capCmt.Merge()\n\n\tif err := capmerge.SetWith(c.merge.with); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := capmerge.SetHead(c.merge.head); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &capCmt, nil\n}\n\n// FromCapnp will set the content of `msg` into the commit,\n// overwriting any previous state.\nfunc (c *Commit) FromCapnp(msg *capnp.Message) error {\n\tcapNd, err := capnp_model.ReadRootNode(msg)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn c.FromCapnpNode(capNd)\n}\n\n// FromCapnpNode converts a serialized node to a normal node.\nfunc (c *Commit) FromCapnpNode(capNd capnp_model.Node) error {\n\tif err := c.parseBaseAttrsFromNode(capNd); err != nil {\n\t\treturn err\n\t}\n\n\tc.nodeType = NodeTypeCommit\n\tcapCmt, err := capNd.Commit()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn c.readCommitAttrs(capCmt)\n}\n\nfunc (c *Commit) readCommitAttrs(capCmt capnp_model.Commit) error {\n\tvar err error\n\n\tc.author, err = capCmt.Author()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tc.message, err = capCmt.Message()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tc.root, err = capCmt.Root()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tc.parent, err = capCmt.Parent()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tc.index = capCmt.Index()\n\n\tcapMerge := capCmt.Merge()\n\tc.merge.head, err = capMerge.Head()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tc.merge.with, err = capMerge.With()\n\treturn err\n}\n\n// IsBoxed will return True if the ommit was already boxed\n// (i.e. is a finished commit and no staging commit)\nfunc (c *Commit) IsBoxed() bool {\n\treturn c.tree != nil\n}\n\n// padHash will take a Hash and pad it's representation to 2048 bytes.\n// This is done so we can support different hash sizes later on.\n// We need fixed lengths for the hash calculation of a commit.\nfunc padHash(hash h.Hash) []byte {\n\tpadded := make([]byte, 2048)\n\tcopy(padded, hash.Bytes())\n\treturn padded\n}\n\n// Root returns the current root hash\n// You shall not modify the returned hash.\nfunc (c *Commit) Root() h.Hash {\n\treturn c.root\n}\n\n// SetRoot sets the root directory of this commit.\nfunc (c *Commit) SetRoot(hash h.Hash) {\n\tc.root = hash.Clone()\n}\n\n// BoxCommit takes all currently filled data and calculates the final hash.\n// It also will update the modification time.\n// Only a boxed commit should be\nfunc (c *Commit) BoxCommit(author string, message string) error {\n\tif c.root == nil {\n\t\treturn fmt.Errorf(\"Cannot box commit: root directory is empty\")\n\t}\n\n\tc.author = author\n\n\tbuf := &bytes.Buffer{}\n\n\t// If parent == nil, this will be EmptyBackendHash.\n\tbuf.Write(padHash(c.parent))\n\n\t// Write the root hash.\n\tbuf.Write(padHash(c.root))\n\n\t// Write the author hash. Different author -> different content.\n\tbuf.Write(padHash(h.Sum([]byte(c.author))))\n\n\t// Write the message last, it may be arbitrary length.\n\tbuf.Write([]byte(message))\n\n\tmh := h.Sum(buf.Bytes())\n\tc.message = message\n\tc.tree = h.Hash(mh)\n\treturn nil\n}\n\n// String will return a nice representation of a commit.\nfunc (c *Commit) String() string {\n\treturn fmt.Sprintf(\n\t\t\"\",\n\t\tc.tree.B58String(),\n\t\tc.message,\n\t)\n}\n\n// SetMergeMarker remembers that we merged with the user `with`\n// at this commit at `remoteHead`.\nfunc (c *Commit) SetMergeMarker(with string, remoteHead h.Hash) {\n\tc.merge.with = with\n\tc.merge.head = remoteHead.Clone()\n}\n\n// MergeMarker returns the merge info for this commit, if any.\nfunc (c *Commit) MergeMarker() (string, h.Hash) {\n\treturn c.merge.with, c.merge.head\n}\n\n// /////////////////// METADATA INTERFACE ///////////////////\n\n// Name will return the hash of the commit.\nfunc (c *Commit) Name() string {\n\treturn c.tree.B58String()\n}\n\n// Message will return the commit message of this commit\nfunc (c *Commit) Message() string {\n\treturn c.message\n}\n\n// Path will return the path of the commit, which will\nfunc (c *Commit) Path() string {\n\treturn prefixSlash(path.Join(\".snapshots\", c.Name()))\n}\n\n// Size will always return 0 since a commit has no defined size.\n// If you're interested in the size of the snapshot, check the size\n// of the root directory.\nfunc (c *Commit) Size() uint64 {\n\treturn 0\n}\n\n// CachedSize returns zero.\n// Same reasons as for Size() above.\nfunc (c *Commit) CachedSize() int64 { return 0 }\n\n// Index of the commit. First commit has the index 0,\n// next commit has the index 1 and so on.\nfunc (c *Commit) Index() int64 {\n\treturn c.index\n}\n\n/////////////// HIERARCHY INTERFACE ///////////////\n\n// NChildren will always return 1, since a commit has always exactly one\n// root dir attached.\nfunc (c *Commit) NChildren() int {\n\treturn 1\n}\n\n// Child will return the root directory, no matter what name is given.\nfunc (c *Commit) Child(lkr Linker, _ string) (Node, error) {\n\t// Return the root directory, no matter what name was passed.\n\treturn lkr.NodeByHash(c.root)\n}\n\n// Parent will return the parent commit of this node or nil\n// if it is the first commit ever made.\nfunc (c *Commit) Parent(lkr Linker) (Node, error) {\n\tif c.parent == nil {\n\t\treturn nil, nil\n\t}\n\n\treturn lkr.NodeByHash(c.parent)\n}\n\n// SetParent sets the parent of the commit to `nd`.\nfunc (c *Commit) SetParent(lkr Linker, nd Node) error {\n\tc.parent = nd.TreeHash().Clone()\n\treturn nil\n}\n\n// SetModTime sets the commits modtime to `t`.\n// This should only be used for the most recent commit.\nfunc (c *Commit) SetModTime(t time.Time) {\n\tc.modTime = t\n}\n\n// Assert that Commit follows the Node interface:\nvar _ Node = &Commit{}\n" }, { "path": "catfs/nodes/commit_test.go", "content": "package nodes\n\nimport (\n\t\"testing\"\n\n\th \"github.com/sahib/brig/util/hashlib\"\n\t\"github.com/stretchr/testify/require\"\n\tcapnp \"zombiezen.com/go/capnproto2\"\n)\n\nfunc TestCommit(t *testing.T) {\n\tcmt, err := NewEmptyCommit(0, 42)\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to create commit: %v\", err)\n\t}\n\n\tcmt.root = h.EmptyBackendHash\n\tcmt.parent = h.EmptyBackendHash\n\tcmt.Base.name = \"some commit\"\n\n\tcmt.SetMergeMarker(AuthorOfStage, h.TestDummy(t, 42))\n\n\tif err := cmt.BoxCommit(AuthorOfStage, \"Hello\"); err != nil {\n\t\tt.Fatalf(\"Failed to box commit: %v\", err)\n\t}\n\n\tmsg, err := cmt.ToCapnp()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to convert commit to capnp: %v\", err)\n\t}\n\n\tdata, err := msg.Marshal()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to marshal message: %v\", err)\n\t}\n\n\tnewMsg, err := capnp.Unmarshal(data)\n\tif err != nil {\n\t\tt.Fatalf(\"Unmarshal failed: %v\", err)\n\t}\n\n\tempty := &Commit{}\n\tif err := empty.FromCapnp(newMsg); err != nil {\n\t\tt.Fatalf(\"From failed: %v\", err)\n\t}\n\n\tif empty.message != \"Hello\" {\n\t\tt.Fatalf(\"Bad message unmarshaled: %v\", empty.message)\n\t}\n\n\tif !empty.root.Equal(h.EmptyBackendHash) {\n\t\tt.Fatalf(\"Bad root unmarshaled: %v\", empty.root)\n\t}\n\n\tif !empty.parent.Equal(h.EmptyBackendHash) {\n\t\tt.Fatalf(\"Bad parent unmarshaled: %v\", empty.root)\n\t}\n\n\tif empty.author != AuthorOfStage {\n\t\tt.Fatalf(\"Bad author unmarshaled: %v\", empty.root)\n\t}\n\n\tif empty.index != 42 {\n\t\tt.Fatalf(\"Index did not match in umarshalled: %v\", empty.index)\n\t}\n\n\tperson, remoteHead := empty.MergeMarker()\n\tif !remoteHead.Equal(h.TestDummy(t, 42)) {\n\t\tt.Fatalf(\"Remote head was not loaded correctly: %v\", remoteHead.Bytes())\n\t}\n\n\tif person != AuthorOfStage {\n\t\tt.Fatalf(\"Person from unmarshaled commit does not equal staging author: %v\", person)\n\t}\n\n\tempty.modTime = cmt.modTime\n\trequire.Equal(t, empty, cmt)\n}\n" }, { "path": "catfs/nodes/directory.go", "content": "package nodes\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"path\"\n\t\"sort\"\n\t\"strings\"\n\t\"time\"\n\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tcapnp_model \"github.com/sahib/brig/catfs/nodes/capnp\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\tcapnp \"zombiezen.com/go/capnproto2\"\n)\n\n// Directory is a typical directory that may contain\n// several other directories or files.\ntype Directory struct {\n\tBase\n\n\tsize uint64\n\tcachedSize int64 // Negative indicates that it is unknown\n\tparentName string\n\tchildren map[string]h.Hash\n\tcontents map[string]h.Hash\n\torder []string\n}\n\n// NewEmptyDirectory creates a new empty directory that does not exist yet.\nfunc NewEmptyDirectory(\n\tlkr Linker, parent *Directory, name string, user string, inode uint64,\n) (*Directory, error) {\n\tabsPath := \"\"\n\tif parent != nil {\n\t\tabsPath = path.Join(parent.Path(), name)\n\t}\n\n\tnewDir := &Directory{\n\t\tBase: Base{\n\t\t\tinode: inode,\n\t\t\tuser: user,\n\t\t\ttree: h.Sum([]byte(absPath)),\n\t\t\tcontent: h.EmptyInternalHash.Clone(),\n\t\t\tbackend: h.EmptyBackendHash.Clone(),\n\t\t\tname: name,\n\t\t\tnodeType: NodeTypeDirectory,\n\t\t\tmodTime: time.Now().Truncate(time.Microsecond),\n\t\t},\n\t\tchildren: make(map[string]h.Hash),\n\t\tcontents: make(map[string]h.Hash),\n\t\torder: []string{},\n\t}\n\n\tif parent != nil {\n\t\t// parentName is set by Add:\n\t\tif err := parent.Add(lkr, newDir); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn newDir, nil\n}\n\nfunc (d *Directory) String() string {\n\treturn fmt.Sprintf(\"\", d.Path(), d.TreeHash(), d.Inode())\n}\n\n// ToCapnp converts the directory to an easily serializable capnp message.\nfunc (d *Directory) ToCapnp() (*capnp.Message, error) {\n\tmsg, seg, err := capnp.NewMessage(capnp.SingleSegment(nil))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapNd, err := capnp_model.NewRootNode(seg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn msg, d.ToCapnpNode(seg, capNd)\n}\n\n// ToCapnpNode converts this node to a serializable capnp proto node.\nfunc (d *Directory) ToCapnpNode(seg *capnp.Segment, capNd capnp_model.Node) error {\n\tif err := d.setBaseAttrsToNode(capNd); err != nil {\n\t\treturn err\n\t}\n\n\tcapDir, err := d.setDirectoryAttrs(seg)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn capNd.SetDirectory(*capDir)\n}\n\nfunc (d *Directory) setDirectoryAttrs(seg *capnp.Segment) (*capnp_model.Directory, error) {\n\tcapDir, err := capnp_model.NewDirectory(seg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tchildren, err := capnp_model.NewDirEntry_List(seg, int32(len(d.children)))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tentryIdx := 0\n\tfor name, hash := range d.children {\n\t\tentry, err := capnp_model.NewDirEntry(seg)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif err := entry.SetName(name); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tif err := entry.SetHash(hash); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tif err := children.Set(entryIdx, entry); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tentryIdx++\n\t}\n\n\tif err := capDir.SetChildren(children); err != nil {\n\t\treturn nil, err\n\t}\n\n\tcontents, err := capnp_model.NewDirEntry_List(seg, int32(len(d.contents)))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tentryIdx = 0\n\tfor name, hash := range d.contents {\n\t\tentry, err := capnp_model.NewDirEntry(seg)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif err := entry.SetName(name); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tif err := entry.SetHash(hash); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tif err := contents.Set(entryIdx, entry); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tentryIdx++\n\t}\n\n\tif err := capDir.SetContents(contents); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := capDir.SetParent(d.parentName); err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapDir.SetSize(d.size)\n\tcapDir.SetCachedSize(int64(d.size))\n\treturn &capDir, nil\n}\n\n// FromCapnp will take the result of ToCapnp and set all of it's attributes.\nfunc (d *Directory) FromCapnp(msg *capnp.Message) error {\n\tcapNd, err := capnp_model.ReadRootNode(msg)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn d.FromCapnpNode(capNd)\n}\n\n// FromCapnpNode converts a serialized node to a normal node.\nfunc (d *Directory) FromCapnpNode(capNd capnp_model.Node) error {\n\tif err := d.parseBaseAttrsFromNode(capNd); err != nil {\n\t\treturn err\n\t}\n\n\tcapDir, err := capNd.Directory()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn d.readDirectoryAttr(capDir)\n}\n\nfunc (d *Directory) readDirectoryAttr(capDir capnp_model.Directory) error {\n\tvar err error\n\n\td.size = capDir.Size()\n\td.cachedSize = capDir.CachedSize()\n\td.parentName, err = capDir.Parent()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tchildList, err := capDir.Children()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\td.children = make(map[string]h.Hash)\n\tfor i := 0; i < childList.Len(); i++ {\n\t\tentry := childList.At(i)\n\t\tname, err := entry.Name()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\thash, err := entry.Hash()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\td.children[name] = hash\n\t\td.order = append(d.order, name)\n\t}\n\n\tcontentList, err := capDir.Contents()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\td.contents = make(map[string]h.Hash)\n\tfor i := 0; i < contentList.Len(); i++ {\n\t\tentry := contentList.At(i)\n\t\tname, err := entry.Name()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\thash, err := entry.Hash()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\td.contents[name] = hash\n\t}\n\n\tsort.Strings(d.order)\n\td.nodeType = NodeTypeDirectory\n\treturn nil\n}\n\n////////////// NODE INTERFACE /////////////////\n\n// Name returns the dirname of this directory.\nfunc (d *Directory) Name() string {\n\treturn d.name\n}\n\n// Size returns the accumulated size of the directory\n// (i.e. the sum of a files in it, excluding ghosts)\nfunc (d *Directory) Size() uint64 {\n\treturn d.size\n}\n\n// CachedSize is similar to Size() above but for accumulated backends storage\nfunc (d *Directory) CachedSize() int64 {\n\treturn d.cachedSize\n}\n\n// Path returns the full path of this node.\nfunc (d *Directory) Path() string {\n\treturn prefixSlash(path.Join(d.parentName, d.Base.name))\n}\n\n// NChildren returns the number of children the directory has.\nfunc (d *Directory) NChildren() int {\n\treturn len(d.children)\n}\n\n// Child returns a specific child with `name` or nil, if it was not found.\nfunc (d *Directory) Child(lkr Linker, name string) (Node, error) {\n\tchildHash, ok := d.children[name]\n\tif !ok {\n\t\treturn nil, nil\n\t}\n\n\treturn lkr.NodeByHash(childHash)\n}\n\n// Parent will return the parent of this directory or nil,\n// if this directory is already the root directory.\nfunc (d *Directory) Parent(lkr Linker) (Node, error) {\n\tif d.parentName == \"\" {\n\t\treturn nil, nil\n\t}\n\n\treturn lkr.LookupNode(d.parentName)\n}\n\n// SetParent will set the parent of this directory to `nd`.\nfunc (d *Directory) SetParent(lkr Linker, nd Node) error {\n\tif d.Path() == \"/\" {\n\t\treturn nil\n\t}\n\n\tif nd == nil {\n\t\td.parentName = \"\"\n\t} else {\n\t\td.parentName = nd.Path()\n\t}\n\n\treturn nil\n}\n\n// ////////////// TREE MOVEMENT /////////////////\n\n// VisitChildren will call `fn` for each of it's direct children.\n// The order of visits is lexicographical based on the child name.\nfunc (d *Directory) VisitChildren(lkr Linker, fn func(nd Node) error) error {\n\tfor _, name := range d.order {\n\t\thash := d.children[name]\n\t\tchild, err := lkr.NodeByHash(hash)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif child == nil {\n\t\t\treturn fmt.Errorf(\"BUG: dead link in tree: %s => %s\", name, hash.B58String())\n\t\t}\n\n\t\tif err := fn(child); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// ChildrenSorted returns a list of children node objects, sorted lexically by\n// their path. Use this whenever you want to have a defined order of nodes,\n// but do not really care what order.\nfunc (d *Directory) ChildrenSorted(lkr Linker) ([]Node, error) {\n\tchildren := []Node{}\n\terr := d.VisitChildren(lkr, func(nd Node) error {\n\t\tchildren = append(children, nd)\n\t\treturn nil\n\t})\n\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn children, nil\n}\n\n// Up will call `visit` for each node onto the way top to the root node,\n// including this directory.\nfunc (d *Directory) Up(lkr Linker, visit func(par *Directory) error) error {\n\troot, err := lkr.Root()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\telems := strings.Split(d.Path(), \"/\")\n\tdirs := []*Directory{root}\n\tcurr := root\n\n\tfor _, elem := range elems {\n\t\tif elem == \"\" {\n\t\t\tcontinue\n\t\t}\n\n\t\tchildHash, ok := curr.children[elem]\n\t\tif !ok {\n\t\t\t// This usually means that some link is missing.\n\t\t\treturn fmt.Errorf(\"bug: cannot reach self from root in up()\")\n\t\t}\n\n\t\tchildNode, err := lkr.NodeByHash(childHash)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tchild, ok := childNode.(*Directory)\n\t\tif !ok {\n\t\t\treturn fmt.Errorf(\"bug: non-directory in up(): %v\", childHash)\n\t\t}\n\n\t\tdirs = append(dirs, child)\n\t\tcurr = child\n\t}\n\n\t// Visit the nodes in reverse order, self first, root last:\n\tfor idx := len(dirs) - 1; idx >= 0; idx-- {\n\t\tif err := visit(dirs[idx]); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// IsRoot returns true if this directory is the root directory.\nfunc (d *Directory) IsRoot() bool {\n\treturn d.parentName == \"\"\n}\n\n// ErrSkipChild can be returned inside a Walk() closure to stop descending\n// recursively into a directory.\nvar ErrSkipChild = errors.New(\"skip sub directory\")\n\n// Walk calls `visit` for each node below `node`, including `node`.\n// If `dfs` is true, depth first search will be used.\n// If `dfs` is false, breadth first search will be used.\n// It is valid to pass a File to Walk(), then visit will be called exactly once.\n//\n// It is possible to return the special error value ErrSkipChild in the callback.\n// In this case, the children of this node are skipped.\n// For this to work, `dfs` has to be false.\nfunc Walk(lkr Linker, node Node, dfs bool, visit func(child Node) error) error {\n\tif node == nil {\n\t\treturn nil\n\t}\n\n\tif node.Type() != NodeTypeDirectory {\n\t\terr := visit(node)\n\t\tif err == ErrSkipChild {\n\t\t\treturn nil\n\t\t}\n\n\t\treturn err\n\t}\n\n\td, ok := node.(*Directory)\n\tif !ok {\n\t\treturn ie.ErrBadNode\n\t}\n\n\tif !dfs {\n\t\tif err := visit(node); err != nil {\n\t\t\tif err == ErrSkipChild {\n\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\treturn err\n\t\t}\n\t}\n\n\tfor _, name := range d.order {\n\t\thash := d.children[name]\n\t\tchild, err := lkr.NodeByHash(hash)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif child == nil {\n\t\t\treturn fmt.Errorf(\"walk: could not resolve %s (%s)\", name, hash.B58String())\n\t\t}\n\n\t\tif err := Walk(lkr, child, dfs, visit); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tif dfs {\n\t\tif err := visit(node); err != nil {\n\t\t\tif err == ErrSkipChild {\n\t\t\t\tpanic(\"bug: you cannot use dfs=true and ErrSkipChild together\")\n\t\t\t}\n\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// Lookup will lookup `repoPath` relative to this directory.\nfunc (d *Directory) Lookup(lkr Linker, repoPath string) (Node, error) {\n\trepoPath = prefixSlash(path.Clean(repoPath))\n\telems := strings.Split(repoPath, \"/\")\n\n\t// Strip off the first empty field:\n\telems = elems[1:]\n\n\tif len(elems) == 1 && elems[0] == \"\" {\n\t\treturn d, nil\n\t}\n\n\tvar curr Node = d\n\tvar err error\n\n\tfor idx, elem := range elems {\n\t\tcurr, err = curr.Child(lkr, elem)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif curr == nil {\n\t\t\treturn nil, ie.NoSuchFile(repoPath)\n\t\t}\n\n\t\t// If the child is a ghost and we did not fully resolve the path\n\t\t// yet we stop here. If it's the ghost of a directory we could\n\t\t// resolve its children, but that would be confusing.\n\t\tif curr.Type() == NodeTypeGhost && idx != len(elems)-1 {\n\t\t\treturn nil, ie.NoSuchFile(repoPath)\n\t\t}\n\t}\n\n\treturn curr, nil\n}\n\n//////////// STATE ALTERING METHODS //////////////\n\n// SetSize sets the size of this directory.\nfunc (d *Directory) SetSize(size uint64) { d.size = size }\n\n// SetCachedSize sets the cached size of the directory.\nfunc (d *Directory) SetCachedSize(cachedSize int64) { d.cachedSize = cachedSize }\n\n// SetName will set the name of this directory.\nfunc (d *Directory) SetName(name string) {\n\td.name = name\n}\n\n// SetModTime will set a new mod time to this directory (i.e. \"touch\" it)\nfunc (d *Directory) SetModTime(modTime time.Time) {\n\td.Base.modTime = modTime.Truncate(time.Microsecond)\n}\n\n// Copy returns a copy of the directory with `inode` changed.\nfunc (d *Directory) Copy(inode uint64) ModNode {\n\tchildren := make(map[string]h.Hash)\n\tcontents := make(map[string]h.Hash)\n\n\tfor name, hash := range d.children {\n\t\tchildren[name] = hash.Clone()\n\t}\n\n\tfor name, hash := range d.contents {\n\t\tcontents[name] = hash.Clone()\n\t}\n\n\torder := make([]string, len(d.order))\n\tcopy(order, d.order)\n\n\treturn &Directory{\n\t\tBase: d.Base.copyBase(inode),\n\t\tsize: d.size,\n\t\tparentName: d.parentName,\n\t\tchildren: children,\n\t\tcontents: contents,\n\t\torder: order,\n\t}\n}\n\nfunc (d *Directory) rehash(lkr Linker, updateContentHash bool) error {\n\tnewTreeHash := h.Sum([]byte(path.Join(d.parentName, d.name)))\n\tnewContentHash := h.EmptyInternalHash.Clone()\n\tfor _, name := range d.order {\n\t\tnewTreeHash = newTreeHash.Mix(d.children[name])\n\n\t\tif childContent := d.contents[name]; updateContentHash && childContent != nil {\n\t\t\t// The child content might be nil in case of ghost.\n\t\t\t// Those should not add to the content calculation.\n\t\t\tnewContentHash = newContentHash.Mix(childContent)\n\t\t}\n\t}\n\n\toldHash := d.tree.Clone()\n\td.tree = newTreeHash\n\n\tif updateContentHash {\n\t\td.content = newContentHash\n\t}\n\n\tlkr.MemIndexSwap(d, oldHash, true)\n\treturn nil\n}\n\n// Add `nd` to this directory using `lkr`.\nfunc (d *Directory) Add(lkr Linker, nd Node) error {\n\tif nd == d {\n\t\treturn fmt.Errorf(\"bug: attempting to add `%s` to itself\", nd.Path())\n\t}\n\n\tif _, ok := d.children[nd.Name()]; ok {\n\t\ttwin, err := d.Child(lkr, nd.Name())\n\t\tif err != nil {\n\t\t\treturn ie.ErrExists\n\t\t}\n\t\tif twin.Type() != NodeTypeGhost {\n\t\t\treturn ie.ErrExists\n\t\t}\n\t\t// Twin is a ghost. We delete it to clear space for a new (added) node.\n\t\terr = d.RemoveChild(lkr, twin)\n\t\tif err != nil {\n\t\t\t// the ghost twin stays and we report it as existing\n\t\t\treturn ie.ErrExists\n\t\t}\n\t}\n\n\tnodeSize := nd.Size()\n\tnodeCachedSize := nd.CachedSize()\n\tnodeHash := nd.TreeHash()\n\tnodeContent := nd.ContentHash()\n\n\td.children[nd.Name()] = nodeHash\n\tif nd.Type() != NodeTypeGhost {\n\t\td.contents[nd.Name()] = nodeContent\n\t}\n\n\tnameIdx := sort.SearchStrings(d.order, nd.Name())\n\tsuffix := append([]string{nd.Name()}, d.order[nameIdx:]...)\n\td.order = append(d.order[:nameIdx], suffix...)\n\n\tvar lastNd Node\n\terr := d.Up(lkr, func(parent *Directory) error {\n\t\tif nd.Type() != NodeTypeGhost {\n\t\t\t// Only add to the size if it's not a ghost.\n\t\t\t// They do not really count as size.\n\t\t\t// Same goes for the node content.\n\t\t\tparent.size += nodeSize\n\t\t\tparent.cachedSize += nodeCachedSize\n\n\t\t}\n\n\t\tif lastNd != nil {\n\t\t\tparent.children[lastNd.Name()] = lastNd.TreeHash()\n\n\t\t\tif nd.Type() != NodeTypeGhost {\n\t\t\t\tparent.contents[lastNd.Name()] = lastNd.ContentHash()\n\t\t\t}\n\t\t}\n\n\t\tif err := parent.rehash(lkr, true); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tlastNd = parent\n\t\treturn nil\n\t})\n\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Establish the link between parent and child:\n\treturn nd.SetParent(lkr, d)\n}\n\n// RemoveChild removes the child named `name` from it's children.\n// There is no way to remove the root node.\nfunc (d *Directory) RemoveChild(lkr Linker, nd Node) error {\n\tname := nd.Name()\n\tif _, ok := d.children[name]; !ok {\n\t\treturn ie.NoSuchFile(name)\n\t}\n\n\t// Unset parent from child:\n\tif err := nd.SetParent(lkr, nil); err != nil {\n\t\treturn err\n\t}\n\n\t// Delete it from orders and children.\n\t// This assumes that it definitely was part of orders before.\n\tdelete(d.children, name)\n\tdelete(d.contents, name)\n\n\tnameIdx := sort.SearchStrings(d.order, name)\n\td.order = append(d.order[:nameIdx], d.order[nameIdx+1:]...)\n\n\tvar lastNd Node\n\tnodeSize := nd.Size()\n\tnodeCachedSize := nd.CachedSize()\n\treturn d.Up(lkr, func(parent *Directory) error {\n\t\tif nd.Type() != NodeTypeGhost {\n\t\t\tparent.size -= nodeSize\n\t\t\tparent.cachedSize -= nodeCachedSize\n\t\t}\n\n\t\tif lastNd != nil {\n\t\t\tparent.children[lastNd.Name()] = lastNd.TreeHash()\n\n\t\t\tif nd.Type() != NodeTypeGhost {\n\t\t\t\tparent.contents[lastNd.Name()] = lastNd.ContentHash()\n\t\t\t}\n\t\t}\n\n\t\tif err := parent.rehash(lkr, true); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tlastNd = parent\n\t\treturn nil\n\t})\n}\n\nfunc (d *Directory) rebuildOrderCache() {\n\td.order = []string{}\n\tfor name := range d.children {\n\t\td.order = append(d.order, name)\n\t}\n\tsort.Strings(d.order)\n}\n\n// NotifyMove should be called whenever a node is being moved.\nfunc (d *Directory) NotifyMove(lkr Linker, newParent *Directory, newPath string) error {\n\tvisited := map[string]Node{}\n\toldRootPath := d.Path()\n\n\terr := Walk(lkr, d, true, func(child Node) error {\n\t\toldChildPath := child.Path()\n\t\tnewChildPath := path.Join(newPath, oldChildPath[len(oldRootPath):])\n\t\tvisited[newChildPath] = child\n\n\t\tswitch child.Type() {\n\t\tcase NodeTypeDirectory:\n\t\t\tchildDir, ok := child.(*Directory)\n\t\t\tif !ok {\n\t\t\t\treturn ie.ErrBadNode\n\t\t\t}\n\n\t\t\tfor name := range childDir.children {\n\t\t\t\tmovedChildPath := path.Join(newChildPath, name)\n\t\t\t\tchildDir.children[name] = visited[movedChildPath].TreeHash()\n\t\t\t}\n\n\t\t\tif err := childDir.rehash(lkr, false); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tdirname, basename := path.Split(newChildPath)\n\t\t\tchildDir.parentName = dirname\n\t\t\tchildDir.SetName(basename)\n\t\t\treturn nil\n\t\tcase NodeTypeFile:\n\t\t\tchildFile, ok := child.(*File)\n\t\t\tif !ok {\n\t\t\t\treturn ie.ErrBadNode\n\t\t\t}\n\n\t\t\tif err := childFile.NotifyMove(lkr, nil, newChildPath); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\tcase NodeTypeGhost:\n\t\t\tchildGhost, ok := child.(*Ghost)\n\t\t\tif !ok {\n\t\t\t\treturn ie.ErrBadNode\n\t\t\t}\n\n\t\t\tchildGhost.SetGhostPath(newChildPath)\n\t\tdefault:\n\t\t\treturn fmt.Errorf(\"bad node type in NotifyMove(): %d\", child.Type())\n\t\t}\n\n\t\treturn nil\n\t})\n\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Fixup the links from the parents to the children:\n\tfor nodePath, node := range visited {\n\t\tif parent, ok := visited[path.Dir(nodePath)]; ok {\n\t\t\tparentDir := parent.(*Directory)\n\t\t\tparentDir.children[path.Base(nodePath)] = node.TreeHash()\n\t\t\tparentDir.rebuildOrderCache()\n\t\t}\n\t}\n\n\tif err := newParent.Add(lkr, d); err != nil {\n\t\treturn err\n\t}\n\n\tnewParent.rebuildOrderCache()\n\treturn nil\n}\n\n// SetUser sets the user that last modified the directory.\nfunc (d *Directory) SetUser(user string) {\n\td.Base.user = user\n}\n\n// Assert that Directory follows the Node interface:\nvar _ ModNode = &Directory{}\n" }, { "path": "catfs/nodes/directory_test.go", "content": "package nodes\n\nimport (\n\t\"testing\"\n\n\tie \"github.com/sahib/brig/catfs/errors\"\n\t\"github.com/stretchr/testify/require\"\n\tcapnp \"zombiezen.com/go/capnproto2\"\n)\n\nfunc TestDirectoryBasics(t *testing.T) {\n\tlkr := NewMockLinker()\n\trepoDir, err := NewEmptyDirectory(lkr, nil, \"\", \"a\", 1)\n\tlkr.MemSetRoot(repoDir)\n\tlkr.AddNode(repoDir, true)\n\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to create empty dir: %v\", err)\n\t}\n\n\tsubDir, err := NewEmptyDirectory(lkr, repoDir, \"sub\", \"b\", 2)\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to create empty sub dir: %v\", err)\n\t}\n\tlkr.AddNode(subDir, true)\n\n\tif err := repoDir.Add(lkr, subDir); err != ie.ErrExists {\n\t\tt.Fatalf(\"Adding sub/ to repo/ worked twice: %v\", err)\n\t}\n\n\t// Fake size here.\n\trepoDir.size = 3\n\trepoDir.cachedSize = 3\n\n\tmsg, err := repoDir.ToCapnp()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to convert repo dir to capnp: %v\", err)\n\t}\n\n\tdata, err := msg.Marshal()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to marshal message: %v\", err)\n\t}\n\n\tnewMsg, err := capnp.Unmarshal(data)\n\tif err != nil {\n\t\tt.Fatalf(\"Unmarshal failed: %v\", err)\n\t}\n\n\tempty := &Directory{}\n\tif err := empty.FromCapnp(newMsg); err != nil {\n\t\tt.Fatalf(\"From capnp failed: %v\", err)\n\t}\n\n\tif empty.size != 3 {\n\t\tt.Fatalf(\"Root size was not loaded correctly: %v\", err)\n\t}\n\n\tif empty.parentName != \"\" {\n\t\tt.Fatalf(\"Root parentName as not loaded correctly: %v\", err)\n\t}\n\n\tif empty.Inode() != 1 {\n\t\tt.Fatalf(\"Inode was not loaded correctly: %v != 1\", empty.Inode())\n\t}\n\n\tif subHash, ok := empty.children[\"sub\"]; ok {\n\t\tif !subHash.Equal(subDir.TreeHash()) {\n\t\t\tt.Fatalf(\"Unmarshaled hash differs (!= sub): %v\", subDir.TreeHash())\n\t\t}\n\t} else {\n\t\tt.Fatalf(\"Root children do not contain sub\")\n\t}\n\n\tempty.modTime = repoDir.modTime\n\trequire.Equal(t, empty, repoDir)\n}\n" }, { "path": "catfs/nodes/doc.go", "content": "// Package nodes implements all nodes and defines basic operations on it.\n//\n// It however does not implement any specific database scheme, nor\n// are any operations implemented that require knowledge of other nodes.\n// If knowledge about other nodes is required, the Linker interface needs\n// to be fulfilled by a higher level.\n//\n// The actual core of brig is built upon this package.\n// Any changes here should thus be well thought through.\npackage nodes\n" }, { "path": "catfs/nodes/file.go", "content": "package nodes\n\nimport (\n\t\"fmt\"\n\t\"path\"\n\t\"time\"\n\n\tcapnp_model \"github.com/sahib/brig/catfs/nodes/capnp\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\tcapnp \"zombiezen.com/go/capnproto2\"\n)\n\n// File represents a single file in the repository.\n// It stores all metadata about it and links to the actual data.\ntype File struct {\n\tBase\n\n\tsize uint64\n\tcachedSize int64 // Negative indicates that it is unknown\n\tparent string\n\tkey []byte\n\tisRaw bool\n}\n\n// NewEmptyFile returns a newly created file under `parent`, named `name`.\nfunc NewEmptyFile(parent *Directory, name string, user string, inode uint64) *File {\n\treturn &File{\n\t\tBase: Base{\n\t\t\tname: name,\n\t\t\tuser: user,\n\t\t\tinode: inode,\n\t\t\tmodTime: time.Now().Truncate(time.Microsecond),\n\t\t\tnodeType: NodeTypeFile,\n\t\t},\n\t\tparent: parent.Path(),\n\t}\n}\n\n// ToCapnp converts a file to a capnp message.\nfunc (f *File) ToCapnp() (*capnp.Message, error) {\n\tmsg, seg, err := capnp.NewMessage(capnp.SingleSegment(nil))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapNd, err := capnp_model.NewRootNode(seg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn msg, f.ToCapnpNode(seg, capNd)\n}\n\n// ToCapnpNode converts this node to a serializable capnp proto node.\nfunc (f *File) ToCapnpNode(seg *capnp.Segment, capNd capnp_model.Node) error {\n\tif err := f.setBaseAttrsToNode(capNd); err != nil {\n\t\treturn err\n\t}\n\n\tcapFile, err := f.setFileAttrs(seg)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn capNd.SetFile(*capFile)\n}\n\nfunc (f *File) setFileAttrs(seg *capnp.Segment) (*capnp_model.File, error) {\n\tcapFile, err := capnp_model.NewFile(seg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := capFile.SetParent(f.parent); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := capFile.SetKey(f.key); err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapFile.SetSize(f.size)\n\tcapFile.SetCachedSize(f.cachedSize)\n\tcapFile.SetIsRaw(f.isRaw)\n\treturn &capFile, nil\n}\n\n// FromCapnp sets all state of `msg` into the file.\nfunc (f *File) FromCapnp(msg *capnp.Message) error {\n\tcapNd, err := capnp_model.ReadRootNode(msg)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn f.FromCapnpNode(capNd)\n}\n\n// FromCapnpNode converts a serialized node to a normal node.\nfunc (f *File) FromCapnpNode(capNd capnp_model.Node) error {\n\tif err := f.parseBaseAttrsFromNode(capNd); err != nil {\n\t\treturn err\n\t}\n\n\tcapFile, err := capNd.File()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn f.readFileAttrs(capFile)\n}\n\nfunc (f *File) readFileAttrs(capFile capnp_model.File) error {\n\tvar err error\n\n\tf.parent, err = capFile.Parent()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tf.isRaw = capFile.IsRaw()\n\tf.nodeType = NodeTypeFile\n\tf.size = capFile.Size()\n\tf.cachedSize = capFile.CachedSize()\n\tf.key, err = capFile.Key()\n\treturn err\n}\n\n////////////////// METADATA INTERFACE //////////////////\n\n// Size returns the number of bytes in the file's content.\nfunc (f *File) Size() uint64 { return f.size }\n\n// CachedSize returns the number of bytes in the file's backend storage.\nfunc (f *File) CachedSize() int64 { return f.cachedSize }\n\n////////////////// ATTRIBUTE SETTERS //////////////////\n\n// SetModTime udates the mod time of the file (i.e. \"touch\"es it)\nfunc (f *File) SetModTime(t time.Time) {\n\tf.modTime = t.Truncate(time.Microsecond)\n}\n\n// SetName set the name of the file.\nfunc (f *File) SetName(n string) { f.name = n }\n\n// SetIsRaw sets the isRaw attribute\nfunc (f *File) SetIsRaw(isRaw bool) { f.isRaw = isRaw }\n\n// SetKey updates the key to a new value, taking ownership of the value.\nfunc (f *File) SetKey(k []byte) { f.key = k }\n\n// SetSize will update the size of the file and update it's mod time.\nfunc (f *File) SetSize(s uint64) {\n\tf.size = s\n\tf.SetModTime(time.Now())\n}\n\n// SetCachedSize will update the cached size of the file and update it's mod time.\nfunc (f *File) SetCachedSize(s int64) {\n\tf.cachedSize = s\n\tf.SetModTime(time.Now())\n}\n\n// Copy copies the contents of the file, except `inode`.\nfunc (f *File) Copy(inode uint64) ModNode {\n\tif f == nil {\n\t\treturn nil\n\t}\n\n\tvar copyKey []byte\n\tif f.key != nil {\n\t\tcopyKey = make([]byte, len(f.key))\n\t\tcopy(copyKey, f.key)\n\t}\n\n\treturn &File{\n\t\tBase: f.Base.copyBase(inode),\n\t\tsize: f.size,\n\t\tcachedSize: f.cachedSize,\n\t\tparent: f.parent,\n\t\tkey: copyKey,\n\t}\n}\n\nfunc (f *File) rehash(lkr Linker, newPath string) {\n\toldHash := f.tree.Clone()\n\tvar contentHash h.Hash\n\tif f.Base.content != nil {\n\t\tcontentHash = f.Base.content.Clone()\n\t} else {\n\t\tcontentHash = h.EmptyInternalHash.Clone()\n\t}\n\n\tf.tree = h.Sum([]byte(fmt.Sprintf(\"%s|%s\", newPath, contentHash)))\n\tlkr.MemIndexSwap(f, oldHash, true)\n}\n\n// NotifyMove should be called when the node moved parents.\nfunc (f *File) NotifyMove(lkr Linker, newParent *Directory, newPath string) error {\n\tdirname, basename := path.Split(newPath)\n\tf.SetName(basename)\n\tf.parent = dirname\n\tf.rehash(lkr, newPath)\n\n\tif newParent != nil {\n\t\tif err := newParent.Add(lkr, f); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tnewParent.rebuildOrderCache()\n\t}\n\n\treturn nil\n}\n\n// SetContent will update the hash of the file (and also the mod time)\nfunc (f *File) SetContent(lkr Linker, content h.Hash) {\n\tf.Base.content = content\n\tf.rehash(lkr, f.Path())\n\tf.SetModTime(time.Now())\n}\n\n// SetBackend will update the hash of the file (and also the mod time)\nfunc (f *File) SetBackend(lkr Linker, backend h.Hash) {\n\tf.Base.backend = backend\n\tf.SetModTime(time.Now())\n}\n\nfunc (f *File) String() string {\n\treturn fmt.Sprintf(\n\t\t\"\",\n\t\tf.Path(),\n\t\tf.TreeHash(),\n\t\tf.Inode(),\n\t\tf.IsRaw(),\n\t)\n}\n\n// Path will return the absolute path of the file.\nfunc (f *File) Path() string {\n\treturn prefixSlash(path.Join(f.parent, f.name))\n}\n\n// IsRaw returns if the file is associated with a raw stream.\n// raw streams should not be decoded.\nfunc (f *File) IsRaw() bool { return f.isRaw }\n\n////////////////// HIERARCHY INTERFACE //////////////////\n\n// NChildren returns the number of children this file node has.\nfunc (f *File) NChildren() int {\n\treturn 0\n}\n\n// Child will return always nil, since files don't have children.\nfunc (f *File) Child(_ Linker, name string) (Node, error) {\n\t// A file never has a child. Sad but true.\n\treturn nil, nil\n}\n\n// Parent returns the parent directory of File.\n// If `f` is already the root, it will return itself (and never nil).\nfunc (f *File) Parent(lkr Linker) (Node, error) {\n\treturn lkr.LookupNode(f.parent)\n}\n\n// SetParent will set the parent of the file to `parent`.\nfunc (f *File) SetParent(_ Linker, parent Node) error {\n\tif parent == nil {\n\t\treturn nil\n\t}\n\n\tf.parent = parent.Path()\n\treturn nil\n}\n\n// Key returns the current key of the file.\nfunc (f *File) Key() []byte {\n\treturn f.key\n}\n\n// SetUser sets the user that last modified the file.\nfunc (f *File) SetUser(user string) {\n\tf.Base.user = user\n}\n\n// Interface check for debugging:\nvar _ ModNode = &File{}\nvar _ Streamable = &File{}\n" }, { "path": "catfs/nodes/file_test.go", "content": "package nodes\n\nimport (\n\t\"bytes\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/require\"\n\tcapnp \"zombiezen.com/go/capnproto2\"\n)\n\nfunc TestFile(t *testing.T) {\n\tlkr := NewMockLinker()\n\troot, err := NewEmptyDirectory(lkr, nil, \"\", \"a\", 2)\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to create root dir: %v\", err)\n\t}\n\tlkr.AddNode(root, true)\n\tlkr.MemSetRoot(root)\n\n\tfile := NewEmptyFile(root, \"some file\", \"a\", 3)\n\tlkr.AddNode(file, true)\n\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to create empty file: %v\", err)\n\t}\n\n\tfile.SetName(\"new_name\")\n\tfile.SetKey([]byte{1, 2, 3})\n\tfile.SetSize(42)\n\tfile.SetContent(lkr, []byte{4, 5, 6})\n\tfile.SetBackend(lkr, []byte{7, 8, 9})\n\thashBeforeUnmarshal := file.TreeHash().Clone()\n\n\tnow := time.Now()\n\tfile.SetModTime(now)\n\n\tmsg, err := file.ToCapnp()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to convert repo dir to capnp: %v\", err)\n\t}\n\n\tdata, err := msg.Marshal()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to marshal message: %v\", err)\n\t}\n\n\tnewMsg, err := capnp.Unmarshal(data)\n\tif err != nil {\n\t\tt.Fatalf(\"Unmarshal failed: %v\", err)\n\t}\n\n\tempty := &File{}\n\tif err := empty.FromCapnp(newMsg); err != nil {\n\t\tt.Fatalf(\"From capnp failed: %v\", err)\n\t}\n\n\tif empty.Name() != \"new_name\" {\n\t\tt.Fatalf(\"Name differs after unmarshal: %v\", empty.Name())\n\t}\n\n\tif empty.ModTime() != now.Truncate(time.Microsecond) {\n\t\tt.Fatalf(\"modtime differs after unmarshal: %v Want: %v\", now, empty.ModTime())\n\t}\n\n\tif empty.Size() != 42 {\n\t\tt.Fatalf(\"size differs after unmarshal: %v\", empty.Size())\n\t}\n\n\tif !bytes.Equal(empty.Key(), []byte{1, 2, 3}) {\n\t\tt.Fatalf(\"key differs after unmarshal: %v\", empty.Key())\n\t}\n\n\tif !bytes.Equal(empty.TreeHash(), hashBeforeUnmarshal) {\n\t\tt.Fatalf(\"tree hash differs after unmarshal: %v\", empty.TreeHash())\n\t}\n\n\tif !bytes.Equal(empty.BackendHash(), []byte{7, 8, 9}) {\n\t\tt.Fatalf(\"backend hash differs after unmarshal: %v\", empty.BackendHash())\n\t}\n\n\tif !bytes.Equal(empty.ContentHash(), []byte{4, 5, 6}) {\n\t\tt.Fatalf(\"content hash differs after unmarshal: %v\", empty.ContentHash())\n\t}\n\n\tempty.modTime = file.modTime\n\trequire.Equal(t, empty, file)\n}\n" }, { "path": "catfs/nodes/ghost.go", "content": "package nodes\n\nimport (\n\t\"fmt\"\n\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tcapnp_model \"github.com/sahib/brig/catfs/nodes/capnp\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\tcapnp \"zombiezen.com/go/capnproto2\"\n)\n\n// Ghost is a special kind of Node that marks a moved node.\n// If a file was moved, a ghost will be created for the old place.\n// If another file is moved to the new place, the ghost will be \"resurrected\"\n// with the new content.\ntype Ghost struct {\n\tModNode\n\n\tghostPath string\n\tghostInode uint64\n\toldType NodeType\n}\n\n// MakeGhost takes an existing node and converts it to a ghost.\n// In the ghost form no metadata is lost, but the node should\n// not show up. `inode` will be the new inode of the ghost.\n// It should differ to the previous node.\nfunc MakeGhost(nd ModNode, inode uint64) (*Ghost, error) {\n\tif nd.Type() == NodeTypeGhost {\n\t\tpanic(\"cannot put a ghost in a ghost\")\n\t}\n\n\treturn &Ghost{\n\t\tModNode: nd.Copy(nd.Inode()),\n\t\toldType: nd.Type(),\n\t\tghostInode: inode,\n\t\tghostPath: nd.Path(),\n\t}, nil\n}\n\n// Type always returns NodeTypeGhost\nfunc (g *Ghost) Type() NodeType {\n\treturn NodeTypeGhost\n}\n\n// OldNode returns the node the ghost was when it still was alive.\nfunc (g *Ghost) OldNode() ModNode {\n\treturn g.ModNode\n}\n\n// OldFile returns the file the ghost was when it still was alive.\n// Returns ErrBadNode when it wasn't a file.\nfunc (g *Ghost) OldFile() (*File, error) {\n\tfile, ok := g.ModNode.(*File)\n\tif !ok {\n\t\treturn nil, ie.ErrBadNode\n\t}\n\n\treturn file, nil\n}\n\n// OldDirectory returns the old directory that the node was in lifetime\n// If the ghost was not a directory, ErrBadNode is returned.\nfunc (g *Ghost) OldDirectory() (*Directory, error) {\n\tdirectory, ok := g.ModNode.(*Directory)\n\tif !ok {\n\t\treturn nil, ie.ErrBadNode\n\t}\n\n\treturn directory, nil\n}\n\nfunc (g *Ghost) String() string {\n\treturn fmt.Sprintf(\"\", g.TreeHash(), g.ModNode)\n}\n\n// Path returns the path of the node.\nfunc (g *Ghost) Path() string {\n\treturn g.ghostPath\n}\n\n// TreeHash returns the hash of the node.\nfunc (g *Ghost) TreeHash() h.Hash {\n\treturn h.Sum([]byte(fmt.Sprintf(\"ghost:%s\", g.ModNode.TreeHash())))\n}\n\n// Inode returns the inode\nfunc (g *Ghost) Inode() uint64 {\n\treturn g.ghostInode\n}\n\n// SetGhostPath sets the path of the ghost.\nfunc (g *Ghost) SetGhostPath(newPath string) {\n\tg.ghostPath = newPath\n}\n\n// ToCapnp serializes the underlying node\nfunc (g *Ghost) ToCapnp() (*capnp.Message, error) {\n\tmsg, seg, err := capnp.NewMessage(capnp.SingleSegment(nil))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapNd, err := capnp_model.NewRootNode(seg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn msg, g.ToCapnpNode(seg, capNd)\n}\n\n// ToCapnpNode converts this node to a serializable capnp proto node.\nfunc (g *Ghost) ToCapnpNode(seg *capnp.Segment, capNd capnp_model.Node) error {\n\tvar base *Base\n\tcapghost, err := capNd.NewGhost()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tcapghost.SetGhostInode(g.ghostInode)\n\tif err = capghost.SetGhostPath(g.ghostPath); err != nil {\n\t\treturn err\n\t}\n\n\tswitch g.oldType {\n\tcase NodeTypeFile:\n\t\tfile, ok := g.ModNode.(*File)\n\t\tif !ok {\n\t\t\treturn ie.ErrBadNode\n\t\t}\n\n\t\tcapfile, err := file.setFileAttrs(seg)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tbase = &file.Base\n\t\tif err = capghost.SetFile(*capfile); err != nil {\n\t\t\treturn err\n\t\t}\n\tcase NodeTypeDirectory:\n\t\tdir, ok := g.ModNode.(*Directory)\n\t\tif !ok {\n\t\t\treturn ie.ErrBadNode\n\t\t}\n\n\t\tcapdir, err := dir.setDirectoryAttrs(seg)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tbase = &dir.Base\n\t\tif err = capghost.SetDirectory(*capdir); err != nil {\n\t\t\treturn err\n\t\t}\n\tcase NodeTypeGhost:\n\t\tpanic(\"Recursive ghosts are not possible\")\n\tdefault:\n\t\tpanic(fmt.Sprintf(\"Unknown node type: %d\", g.oldType))\n\t}\n\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif err := base.setBaseAttrsToNode(capNd); err != nil {\n\t\treturn err\n\t}\n\n\treturn capNd.SetGhost(capghost)\n}\n\n// FromCapnp reads all attributes from a previously marshaled ghost.\nfunc (g *Ghost) FromCapnp(msg *capnp.Message) error {\n\tcapNd, err := capnp_model.ReadRootNode(msg)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn g.FromCapnpNode(capNd)\n}\n\n// FromCapnpNode converts a serialized node to a normal node.\nfunc (g *Ghost) FromCapnpNode(capNd capnp_model.Node) error {\n\tif typ := capNd.Which(); typ != capnp_model.Node_Which_ghost {\n\t\treturn fmt.Errorf(\"BUG: ghost unmarshal with non ghost type: %d\", typ)\n\t}\n\n\tcapghost, err := capNd.Ghost()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tg.ghostInode = capghost.GhostInode()\n\tg.ghostPath, err = capghost.GhostPath()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tvar base *Base\n\n\tswitch typ := capghost.Which(); typ {\n\tcase capnp_model.Ghost_Which_directory:\n\t\tcapdir, err := capghost.Directory()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tdir := &Directory{}\n\t\tif err := dir.readDirectoryAttr(capdir); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tg.ModNode = dir\n\t\tg.oldType = NodeTypeDirectory\n\t\tbase = &dir.Base\n\tcase capnp_model.Ghost_Which_file:\n\t\tcapfile, err := capghost.File()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfile := &File{}\n\t\tif err := file.readFileAttrs(capfile); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tg.ModNode = file\n\t\tg.oldType = NodeTypeFile\n\t\tbase = &file.Base\n\tdefault:\n\t\treturn ie.ErrBadNode\n\t}\n\n\treturn base.parseBaseAttrsFromNode(capNd)\n}\n" }, { "path": "catfs/nodes/ghost_test.go", "content": "package nodes\n\nimport (\n\t\"bytes\"\n\t\"testing\"\n\n\th \"github.com/sahib/brig/util/hashlib\"\n\tcapnp \"zombiezen.com/go/capnproto2\"\n)\n\nfunc TestGhost(t *testing.T) {\n\tlkr := NewMockLinker()\n\troot, err := NewEmptyDirectory(lkr, nil, \"\", \"a\", 1)\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to create root dir: %v\", err)\n\t}\n\tlkr.AddNode(root, true)\n\tlkr.MemSetRoot(root)\n\n\tfile := NewEmptyFile(root, \"x.png\", \"a\", 42)\n\tfile.backend = h.TestDummy(t, 2)\n\tfile.tree = h.TestDummy(t, 3)\n\tfile.size = 13\n\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to create empty file: %v\", err)\n\t}\n\n\tghost, err := MakeGhost(file, 666)\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to make root dir a ghost: %v\", err)\n\t}\n\n\tghost.SetGhostPath(\"/other\")\n\n\tif ghost.Type() != NodeTypeGhost {\n\t\tt.Fatalf(\"Ghost does not identify itself as ghost: %d\", ghost.Type())\n\t}\n\n\tif !bytes.Equal(ghost.OldNode().TreeHash(), file.TreeHash()) {\n\t\tt.Fatalf(\"Ghost and real hash differ (%v - %v)\", ghost.TreeHash(), root.TreeHash())\n\t}\n\n\tmsg, err := ghost.ToCapnp()\n\tif err != nil {\n\t\tt.Fatalf(\"Ghost ToCapnp failed: %v\", err)\n\t}\n\n\tdata, err := msg.Marshal()\n\tif err != nil {\n\t\tt.Fatalf(\"Ghost marshal failed: %v\", err)\n\t}\n\n\tnewMsg, err := capnp.Unmarshal(data)\n\tif err != nil {\n\t\tt.Fatalf(\"Ghost unmarshal failed: %v\", err)\n\t}\n\n\tempty := &Ghost{}\n\tif err := empty.FromCapnp(newMsg); err != nil {\n\t\tt.Fatalf(\"Ghost FromCapnp failed: %v\", err)\n\t}\n\n\tif empty.Path() != \"/other\" {\n\t\tt.Fatalf(\"Ghost path was not unmarshaled: %v\", empty.Path())\n\t}\n\n\tif !bytes.Equal(ghost.OldNode().TreeHash(), file.TreeHash()) {\n\t\tt.Fatalf(\"Ghost and real hash differ (%v - %v)\", ghost.TreeHash(), root.TreeHash())\n\t}\n\n\tunmarshaledFile, err := ghost.OldFile()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to cast ghost to old file: %v\", err)\n\t}\n\n\tif !unmarshaledFile.BackendHash().Equal(file.BackendHash()) {\n\t\tt.Fatalf(\"Hash content differs after unmarshal: %v\", unmarshaledFile.BackendHash())\n\t}\n\n\tif !unmarshaledFile.TreeHash().Equal(file.TreeHash()) {\n\t\tt.Fatalf(\"Hash itself differs after unmarshal: %v\", unmarshaledFile.TreeHash())\n\t}\n\n\tif unmarshaledFile.Inode() != file.Inode() {\n\t\tt.Fatalf(\"Inodes differ after unmarshal: %d != %d\", unmarshaledFile.Inode(), file.Inode())\n\t}\n\n\tif unmarshaledFile.Path() != \"/x.png\" {\n\t\tt.Fatalf(\"Path differs after unmarshal: %v\", unmarshaledFile.Path())\n\t}\n\n\tif empty.Inode() != ghost.Inode() {\n\t\tt.Fatalf(\"Inodes differ after unmarshal: %d != %d\", unmarshaledFile.Inode(), file.Inode())\n\t}\n}\n" }, { "path": "catfs/nodes/linker.go", "content": "package nodes\n\nimport (\n\t\"fmt\"\n\n\tie \"github.com/sahib/brig/catfs/errors\"\n\th \"github.com/sahib/brig/util/hashlib\"\n)\n\n// Linker will tell a node how it relates to other nodes\n// and gives it the ability to resolve other nodes by hash.\n// Apart from that it gives the underlying linker implementation\n// the possibility to be notified when a hash changes.\ntype Linker interface {\n\t// Root should return the current root directory.\n\tRoot() (*Directory, error)\n\n\t// LookupNode should resolve `path` starting from the root directory.\n\t// If the path does not exist an error is returned and can be checked\n\t// with IsNoSuchFileError()\n\tLookupNode(path string) (Node, error)\n\n\t// NodeByHash resolves the hash to a specific node.\n\t// If the node does not exist, nil is returned.\n\tNodeByHash(hash h.Hash) (Node, error)\n\n\t// MemIndexSwap should be called when\n\t// the hash of a node changes.\n\tMemIndexSwap(nd Node, oldHash h.Hash, updatePathIndex bool)\n\n\t// MemSetRoot should be called when the current root directory changed.\n\tMemSetRoot(root *Directory)\n}\n\n////////////////////////////\n// MOCKING IMPLEMENTATION //\n////////////////////////////\n\n// MockLinker is supposed to be used for testing.\n// It simply holds all nodes in memory. New nodes should be added via AddNode.\ntype MockLinker struct {\n\troot *Directory\n\tpaths map[string]Node\n\thashes map[string]Node\n}\n\n// NewMockLinker returns a Linker that can be easily used for testing.\nfunc NewMockLinker() *MockLinker {\n\treturn &MockLinker{\n\t\tpaths: make(map[string]Node),\n\t\thashes: make(map[string]Node),\n\t}\n}\n\n// Root returns the currently set root.\n// If none was created yet, an empty directory is returned.\nfunc (ml *MockLinker) Root() (*Directory, error) {\n\tif ml.root != nil {\n\t\treturn ml.root, nil\n\t}\n\n\troot, err := NewEmptyDirectory(ml, nil, \"\", \"\", 0)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tml.root = root\n\treturn root, nil\n}\n\n// LookupNode tries to lookup if there is already a node with this path.\nfunc (ml *MockLinker) LookupNode(path string) (Node, error) {\n\tif node, ok := ml.paths[path]; ok {\n\t\treturn node, nil\n\t}\n\n\treturn nil, ie.NoSuchFile(path)\n}\n\n// NodeByHash will return a previously added node (via AddNode) by it's hash.\nfunc (ml *MockLinker) NodeByHash(hash h.Hash) (Node, error) {\n\tif node, ok := ml.hashes[hash.B58String()]; ok {\n\t\treturn node, nil\n\t}\n\n\treturn nil, fmt.Errorf(\"No such hash\")\n}\n\n// MemSetRoot sets the current root to be `root`.\nfunc (ml *MockLinker) MemSetRoot(root *Directory) {\n\tml.root = root\n}\n\n// MemIndexSwap will replace a node (referenced by `oldHash`) with `nd`.\n// The path does not change.\nfunc (ml *MockLinker) MemIndexSwap(nd Node, oldHash h.Hash, updatePathIndex bool) {\n\tdelete(ml.hashes, oldHash.B58String())\n\tml.AddNode(nd, updatePathIndex)\n}\n\n// AddNode will add a node to the memory index.\n// This is not part of the linker interface.\nfunc (ml *MockLinker) AddNode(nd Node, updatePathIndex bool) {\n\tml.hashes[nd.TreeHash().B58String()] = nd\n\tif updatePathIndex {\n\t\tml.paths[nd.Path()] = nd\n\t}\n}\n" }, { "path": "catfs/nodes/node.go", "content": "package nodes\n\nimport (\n\t\"time\"\n\n\tcapnp_model \"github.com/sahib/brig/catfs/nodes/capnp\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\tcapnp \"zombiezen.com/go/capnproto2\"\n)\n\n// NodeType defines the type of a specific node.\ntype NodeType uint8\n\nconst (\n\t// NodeTypeUnknown should not happen in real programs\n\tNodeTypeUnknown = NodeType(iota)\n\t// NodeTypeFile indicates a regular file\n\tNodeTypeFile\n\t// NodeTypeDirectory indicates a directory\n\tNodeTypeDirectory\n\t// NodeTypeCommit indicates a commit\n\tNodeTypeCommit\n\t// NodeTypeGhost indicates a moved node\n\tNodeTypeGhost\n)\n\nvar nodeTypeToString = map[NodeType]string{\n\tNodeTypeCommit: \"commit\",\n\tNodeTypeGhost: \"ghost\",\n\tNodeTypeFile: \"file\",\n\tNodeTypeDirectory: \"directory\",\n}\n\nfunc (n NodeType) String() string {\n\tif name, ok := nodeTypeToString[n]; ok {\n\t\treturn name\n\t}\n\n\treturn \"unknown\"\n}\n\n// Metadatable is a thing that accumulates certain common node attributes.\ntype Metadatable interface {\n\t// Name returns the name of the object, i.e. the last part of the path,\n\t// which is also commonly called 'basename' in unix filesystems.\n\tName() string\n\n\t// User returns the id of the user that last modified this file.\n\t// (There is no real ownership)\n\tUser() string\n\n\t// Size returns the size of the node in bytes.\n\tSize() uint64\n\n\t// CachedSize returns the size of the node at the backend in bytes.\n\t// Negative indicates unknown status\n\tCachedSize() int64\n\n\t// ModTime returns the time when the last modification to the node happened.\n\tModTime() time.Time\n\n\t// Path of this node.\n\tPath() string\n\n\t// GetType returns the type of the node.\n\tType() NodeType\n\n\t// INode shall return a unique identifier for this node that does\n\t// not change, even when the content of the node changes.\n\tInode() uint64\n\n\t// TreeHash returns the hash value of the node.\n\t//\n\t// It is an error to modify the hash value.\n\t// If you need to modify it, you have to make an own copy via .Clone().\n\tTreeHash() h.Hash\n\n\t// ContentHash is the actual plain text hash of the node.\n\t// This is used for comparing file and directory equality.\n\tContentHash() h.Hash\n\n\t// BackendHash returns the hash under which the stored content\n\t// can be read from the backend.\n\t// It is valid to return nil if the file is empty.\n\tBackendHash() h.Hash\n}\n\n// Serializable is a thing that can be converted to a capnproto message.\ntype Serializable interface {\n\tToCapnp() (*capnp.Message, error)\n\tFromCapnp(*capnp.Message) error\n\n\tToCapnpNode(seg *capnp.Segment, capNd capnp_model.Node) error\n\tFromCapnpNode(capNd capnp_model.Node) error\n}\n\n// HierarchyEntry represents a thing that is placed in\n// a file hierarchy and may have other children beneath it.\ntype HierarchyEntry interface {\n\t// NChildren returns the total number of children to a node.\n\tNChildren() int\n\n\t// Child returns a named child.\n\tChild(lkr Linker, name string) (Node, error)\n\n\t// Parent returns the parent node or nil if there is none.\n\tParent(lkr Linker) (Node, error)\n\n\t// SetParent sets the parent new. Care must be taken to remove old\n\t// references to the node to avoid loops.\n\tSetParent(lkr Linker, nd Node) error\n}\n\n// Streamable represents a thing that can be streamed,\n// given a cryptographic key.\ntype Streamable interface {\n\tKey() []byte\n}\n\n// Node is a single node in brig's MDAG.\n// It is currently either a Commit, a File or a Directory.\ntype Node interface {\n\tMetadatable\n\tSerializable\n\tHierarchyEntry\n}\n\n// ModNode is a node that supports modification of\n// it's core attributes. File and Directory are settable,\n// but a commit is not.\ntype ModNode interface {\n\tNode\n\n\t// SetSize sets the size of the node in bytes\n\tSetSize(size uint64)\n\n\t// SetModTime updates the modtime timestamp\n\tSetModTime(modTime time.Time)\n\n\t// SetName sets the user that last modified the file\n\tSetName(name string)\n\n\t// SetUser sets the user that last modified the file\n\tSetUser(user string)\n\n\t// NotifyMove tells the node that it was moved.\n\t// It should be called whenever the path of the node changed.\n\t// (i.e. not only the name, but parts of the parent path)\n\tNotifyMove(lkr Linker, parent *Directory, newPath string) error\n\n\t// Copy creates a copy of this node with the inode `inode`.\n\tCopy(inode uint64) ModNode\n}\n" }, { "path": "catfs/pinner.go", "content": "package catfs\n\nimport (\n\t\"errors\"\n\n\tcapnp \"github.com/sahib/brig/catfs/capnp\"\n\tc \"github.com/sahib/brig/catfs/core\"\n\t\"github.com/sahib/brig/catfs/db\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\tcapnp_lib \"zombiezen.com/go/capnproto2\"\n)\n\n// errNotPinnedSentinel is returned to signal an early exit in Walk()\nvar errNotPinnedSentinel = errors.New(\"not pinned\")\n\n// pinCacheEntry is one entry in the pin cache.\ntype pinCacheEntry struct {\n\tInodes map[uint64]bool\n}\n\nfunc capnpToPinCacheEntry(data []byte) (*pinCacheEntry, error) {\n\tmsg, err := capnp_lib.Unmarshal(data)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapEntry, err := capnp.ReadRootPinEntry(msg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapPins, err := capEntry.Pins()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tentry := &pinCacheEntry{\n\t\tInodes: make(map[uint64]bool),\n\t}\n\n\tfor idx := 0; idx < capPins.Len(); idx++ {\n\t\tcapPin := capPins.At(idx)\n\t\tentry.Inodes[capPin.Inode()] = capPin.IsPinned()\n\t}\n\n\treturn entry, nil\n}\n\nfunc pinEnryToCapnpData(entry *pinCacheEntry) ([]byte, error) {\n\tmsg, seg, err := capnp_lib.NewMessage(capnp_lib.SingleSegment(nil))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapEntry, err := capnp.NewRootPinEntry(seg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapPinList, err := capnp.NewPin_List(seg, int32(len(entry.Inodes)))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tidx := 0\n\tfor inode, isPinned := range entry.Inodes {\n\t\tcapPin, err := capnp.NewPin(seg)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tcapPin.SetInode(inode)\n\t\tcapPin.SetIsPinned(isPinned)\n\n\t\tif err := capPinList.Set(idx, capPin); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tidx++\n\t}\n\n\tif err := capEntry.SetPins(capPinList); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn msg.Marshal()\n}\n\n// Pinner remembers which hashes are pinned and if they are pinned explicitly.\n// Its API can be used to safely change the pinning state. It assumes that it\n// is the only entitiy the pins & unpins nodes.\ntype Pinner struct {\n\tbk FsBackend\n\tlkr *c.Linker\n}\n\n// NewPinner creates a new pin cache at `pinDbPath`, possibly erroring out.\n// `lkr` and `bk` are used to make PinNode() and UnpinNode() work.\nfunc NewPinner(lkr *c.Linker, bk FsBackend) (*Pinner, error) {\n\treturn &Pinner{lkr: lkr, bk: bk}, nil\n}\n\n// Close the pinning cache.\nfunc (pc *Pinner) Close() error {\n\t// currently a no-op\n\treturn nil\n}\n\nfunc getEntry(kv db.Database, hash h.Hash) (*pinCacheEntry, error) {\n\tdata, err := kv.Get(\"pins\", hash.B58String())\n\tif err != nil {\n\t\tif err == db.ErrNoSuchKey {\n\t\t\treturn nil, nil\n\t\t}\n\n\t\treturn nil, err\n\t}\n\n\treturn capnpToPinCacheEntry(data)\n}\n\n// remember the pin state of a certain hash.\n// This does change anything in the backend but only changes the caching structure.\n// Use with care to avoid data inconsistencies.\nfunc (pc *Pinner) remember(inode uint64, hash h.Hash, isPinned, isExplicit bool) error {\n\treturn pc.lkr.AtomicWithBatch(func(batch db.Batch) (bool, error) {\n\t\toldEntry, err := getEntry(pc.lkr.KV(), hash)\n\t\tif err != nil {\n\t\t\treturn true, err\n\t\t}\n\n\t\tvar inodes map[uint64]bool\n\t\tif oldEntry != nil {\n\t\t\tinodes = oldEntry.Inodes\n\t\t} else {\n\t\t\tinodes = make(map[uint64]bool)\n\t\t}\n\n\t\tif !isPinned {\n\t\t\tdelete(inodes, inode)\n\t\t} else {\n\t\t\tinodes[inode] = isExplicit\n\t\t}\n\n\t\tentry := pinCacheEntry{\n\t\t\tInodes: inodes,\n\t\t}\n\n\t\tdata, err := pinEnryToCapnpData(&entry)\n\t\tif err != nil {\n\t\t\treturn true, err\n\t\t}\n\n\t\tbatch.Put(data, \"pins\", hash.B58String())\n\t\treturn false, nil\n\t})\n}\n\n// IsPinned returns two boolean values indicating the pin status of `inode` and\n// `hash`. If the first value is true, the content is pinned, if the second is\n// true it is pinned explicitly.\nfunc (pc *Pinner) IsPinned(inode uint64, hash h.Hash) (bool, bool, error) {\n\tdata, err := pc.lkr.KV().Get(\"pins\", hash.B58String())\n\tif err != nil && err != db.ErrNoSuchKey {\n\t\treturn false, false, err\n\t}\n\n\tif err == nil {\n\t\t// cache hit\n\t\tentry, err := capnpToPinCacheEntry(data)\n\t\tif err != nil {\n\t\t\treturn false, false, err\n\t\t}\n\n\t\tisExplicit, ok := entry.Inodes[inode]\n\t\treturn ok, isExplicit, nil\n\t}\n\n\t// We do not have this information yet.\n\t// Create a new entry based on the backend information.\n\n\t// silence a key error, ok will be false then.\n\tisPinned, err := pc.bk.IsPinned(hash)\n\tif err != nil {\n\t\treturn false, false, err\n\t}\n\n\t// remember the file to be pinned non-explicitly:\n\tif err := pc.remember(inode, hash, isPinned, false); err != nil {\n\t\treturn false, false, err\n\t}\n\n\treturn isPinned, false, nil\n}\n\n////////////////////////////\n\n// Pin will remember the node at `inode` with hash `hash` as `explicit`ly pinned.\nfunc (pc *Pinner) Pin(inode uint64, hash h.Hash, explicit bool) error {\n\tisPinned, isExplicit, err := pc.IsPinned(inode, hash)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif isPinned {\n\t\tif isExplicit && !explicit {\n\t\t\t// will not \"downgrade\" an existing pin.\n\t\t\treturn nil\n\t\t}\n\t} else {\n\t\tif err := pc.bk.Pin(hash); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn pc.remember(inode, hash, true, explicit)\n}\n\n// Unpin pins the content at `inode` and `hash`. If the pin was explicit,\n// `explicit` must be true to make this work.\nfunc (pc *Pinner) Unpin(inode uint64, hash h.Hash, explicit bool) error {\n\tisPinned, isExplicit, err := pc.IsPinned(inode, hash)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif isPinned {\n\t\tif isExplicit && !explicit {\n\t\t\treturn nil\n\t\t}\n\n\t\tif err := pc.bk.Unpin(hash); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn pc.remember(inode, hash, false, explicit)\n}\n\n////////////////////////////\n\n// doPinOp recursively walks over all children of a node and pins or unpins them.\nfunc (pc *Pinner) doPinOp(op func(uint64, h.Hash, bool) error, nd n.Node, explicit bool) error {\n\treturn n.Walk(pc.lkr, nd, true, func(child n.Node) error {\n\t\tif child.Type() != n.NodeTypeFile {\n\t\t\treturn nil\n\t\t}\n\n\t\tfile, ok := child.(*n.File)\n\t\tif !ok {\n\t\t\treturn ie.ErrBadNode\n\t\t}\n\n\t\treturn op(file.Inode(), file.BackendHash(), explicit)\n\t})\n}\n\n// PinNode tries to pin the node referenced by `nd`.\n// The difference to calling Pin(nd.BackendHash()) is,\n// that this method will pin directories recursively, if given.\n//\n// If the file is already pinned exclusively and you want\n// to pin it non-exclusive, this will be a no-op.\n// In this case you have to unpin it first exclusively.\nfunc (pc *Pinner) PinNode(nd n.Node, explicit bool) error {\n\treturn pc.doPinOp(pc.Pin, nd, explicit)\n}\n\n// UnpinNode is the exact opposite of PinNode.\nfunc (pc *Pinner) UnpinNode(nd n.Node, explicit bool) error {\n\treturn pc.doPinOp(pc.Unpin, nd, explicit)\n}\n\n// IsNodePinned checks if all `nd` is pinned and if so, exlusively.\n// If `nd` is a directory, it will only return true if all children\n// are also pinned (same for second return value).\nfunc (pc *Pinner) IsNodePinned(nd n.Node) (bool, bool, error) {\n\t// Handle special case:\n\t// empty directories should count as pinned.\n\t// (for the sake of the definition that a directory is pinned,\n\t// if all children are also pinned)\n\tif nd.Type() == n.NodeTypeDirectory && nd.NChildren() == 0 {\n\t\treturn true, true, nil\n\t}\n\n\tpinCount := 0\n\texplicitCount := 0\n\ttotalCount := 0\n\n\terr := n.Walk(pc.lkr, nd, true, func(child n.Node) error {\n\t\tif child.Type() != n.NodeTypeFile {\n\t\t\treturn nil\n\t\t}\n\n\t\tfile, ok := child.(*n.File)\n\t\tif !ok {\n\t\t\treturn ie.ErrBadNode\n\t\t}\n\n\t\ttotalCount++\n\n\t\tisPinned, isExplicit, err := pc.IsPinned(file.Inode(), file.BackendHash())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif isExplicit {\n\t\t\texplicitCount++\n\t\t}\n\n\t\tif isPinned {\n\t\t\t// Make sure that we do not count empty directories\n\t\t\t// as pinned nodes.\n\t\t\tpinCount++\n\t\t} else {\n\t\t\t// Return a special error here to stop Walk() iterating.\n\t\t\t// One file is enough to stop IsPinned() from being true.\n\t\t\treturn errNotPinnedSentinel\n\t\t}\n\n\t\treturn nil\n\t})\n\n\tif err != nil && err != errNotPinnedSentinel {\n\t\treturn false, false, err\n\t}\n\n\tif err == errNotPinnedSentinel {\n\t\treturn false, false, nil\n\t}\n\n\treturn pinCount > 0, explicitCount == totalCount, nil\n}\n" }, { "path": "catfs/pinner_test.go", "content": "package catfs\n\nimport (\n\t\"bytes\"\n\t\"testing\"\n\n\tc \"github.com/sahib/brig/catfs/core\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestPinMemCache(t *testing.T) {\n\tc.WithDummyLinker(t, func(lkr *c.Linker) {\n\t\tbackend := NewMemFsBackend()\n\t\tpinner, err := NewPinner(lkr, backend)\n\t\trequire.Nil(t, err)\n\n\t\tcontent := h.TestDummy(t, 1)\n\t\trequire.Nil(t, pinner.remember(1, content, true, false))\n\t\tisPinned, isExplicit, err := pinner.IsPinned(1, content)\n\t\trequire.Nil(t, err)\n\n\t\trequire.True(t, isPinned)\n\t\trequire.False(t, isExplicit)\n\n\t\trequire.Nil(t, pinner.remember(1, content, true, true))\n\t\tisPinned, isExplicit, err = pinner.IsPinned(1, content)\n\t\trequire.Nil(t, err)\n\n\t\trequire.True(t, isPinned)\n\t\trequire.True(t, isExplicit)\n\n\t\trequire.Nil(t, pinner.Close())\n\t})\n}\n\nfunc TestPinRememberHashTwice(t *testing.T) {\n\tc.WithDummyLinker(t, func(lkr *c.Linker) {\n\t\tbackend := NewMemFsBackend()\n\t\tpinner, err := NewPinner(lkr, backend)\n\t\trequire.Nil(t, err)\n\n\t\tcontent := h.TestDummy(t, 1)\n\t\trequire.Nil(t, pinner.remember(1, content, true, false))\n\t\tisPinned, isExplicit, err := pinner.IsPinned(1, content)\n\t\trequire.Nil(t, err)\n\t\trequire.True(t, isPinned)\n\t\trequire.False(t, isExplicit)\n\n\t\trequire.Nil(t, pinner.remember(2, content, true, true))\n\t\tisPinned, isExplicit, err = pinner.IsPinned(2, content)\n\t\trequire.Nil(t, err)\n\t\trequire.True(t, isPinned)\n\t\trequire.True(t, isExplicit)\n\n\t\trequire.Nil(t, pinner.remember(2, content, false, true))\n\t\tisPinned, isExplicit, err = pinner.IsPinned(2, content)\n\t\trequire.Nil(t, err)\n\t\trequire.False(t, isPinned)\n\t\trequire.False(t, isExplicit)\n\n\t\t// old inode is still counted as pinned.\n\t\tisPinned, isExplicit, err = pinner.IsPinned(1, content)\n\t\trequire.Nil(t, err)\n\t\trequire.True(t, isPinned)\n\t\trequire.False(t, isExplicit)\n\n\t\trequire.Nil(t, pinner.Close())\n\t})\n}\n\nfunc TestPinNode(t *testing.T) {\n\twithDummyFS(t, func(fs *FS) {\n\t\trequire.Nil(t, fs.Stage(\"/x\", bytes.NewReader([]byte{1})))\n\t\tx, err := fs.lkr.LookupFile(\"/x\")\n\t\trequire.Nil(t, err)\n\t\trequire.Nil(t, fs.pinner.PinNode(x, false))\n\n\t\tisPinned, isExplicit, err := fs.pinner.IsNodePinned(x)\n\t\trequire.Nil(t, err)\n\t\trequire.True(t, isPinned)\n\t\trequire.False(t, isExplicit)\n\n\t\trequire.Nil(t, fs.pinner.PinNode(x, true))\n\t\tisPinned, isExplicit, err = fs.pinner.IsNodePinned(x)\n\t\trequire.Nil(t, err)\n\t\trequire.True(t, isPinned)\n\t\trequire.True(t, isExplicit)\n\n\t\t// Downgrade unpin(false) when explicit => no change.\n\t\trequire.Nil(t, fs.pinner.UnpinNode(x, false))\n\t\tisPinned, isExplicit, err = fs.pinner.IsNodePinned(x)\n\t\trequire.Nil(t, err)\n\t\trequire.True(t, isPinned)\n\t\trequire.True(t, isExplicit)\n\n\t\trequire.Nil(t, fs.pinner.UnpinNode(x, true))\n\t\tisPinned, isExplicit, err = fs.pinner.IsNodePinned(x)\n\t\trequire.Nil(t, err)\n\t\trequire.False(t, isPinned)\n\t\trequire.False(t, isExplicit)\n\t})\n}\n\nfunc TestPinEntryMarshal(t *testing.T) {\n\tpinEntry := &pinCacheEntry{\n\t\tInodes: map[uint64]bool{\n\t\t\t10: true,\n\t\t\t15: false,\n\t\t\t20: true,\n\t\t},\n\t}\n\n\tdata, err := pinEnryToCapnpData(pinEntry)\n\trequire.Nil(t, err)\n\n\tloadedPinEntry, err := capnpToPinCacheEntry(data)\n\trequire.Nil(t, err)\n\n\trequire.Equal(t, pinEntry, loadedPinEntry)\n}\n\nfunc TestPinEmptyDir(t *testing.T) {\n\twithDummyFS(t, func(fs *FS) {\n\t\trequire.Nil(t, fs.Mkdir(\"/empty\", true))\n\n\t\tdir, err := fs.lkr.LookupDirectory(\"/empty\")\n\t\trequire.Nil(t, err)\n\t\tisPinned, isExplicit, err := fs.pinner.IsNodePinned(dir)\n\t\trequire.Nil(t, err)\n\t\trequire.True(t, isPinned)\n\t\trequire.True(t, isExplicit)\n\t})\n}\n" }, { "path": "catfs/repin.go", "content": "package catfs\n\nimport (\n\t\"sort\"\n\n\t\"github.com/dustin/go-humanize\"\n\te \"github.com/pkg/errors\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\t\"github.com/sahib/brig/catfs/vcs\"\n\t\"github.com/sahib/brig/util\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\ntype partition struct {\n\tPinSize uint64\n\n\t// nodes that are within min_depth and should stay pinned\n\t// (or are even re-pinned if needed)\n\tShouldPin []n.ModNode\n\n\t// nodes that are between min_depth and max_depth.\n\t// they might be unpinned if they exceed the quota.\n\tQuotaCandidates []n.ModNode\n\n\t// nodes that are behind max_depth.\n\t// all of the are unpinned for sure.\n\tDepthCandidates []n.ModNode\n}\n\n// partitionNodeHashes takes all hashes of a node and sorts them into the\n// buckets described in the partition docs.\nfunc (fs *FS) partitionNodeHashes(nd n.ModNode, minDepth, maxDepth int64) (*partition, error) {\n\tcurrDepth := int64(0)\n\tpart := &partition{}\n\n\tcurr, err := fs.lkr.Status()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tseen := make(map[string]bool)\n\twalker := vcs.NewHistoryWalker(fs.lkr, curr, nd)\n\n\tfor walker.Next() {\n\t\tstate := walker.State()\n\t\tcurr := state.Curr\n\n\t\tif curr.Type() == n.NodeTypeGhost {\n\t\t\t// ghosts nodes are always unpinned\n\t\t\tcontinue\n\t\t}\n\n\t\tif seen[curr.BackendHash().B58String()] {\n\t\t\t// We only want to have the first $n distinct versions.\n\t\t\t// Sometimes the versions is duplicated though (removed, readded, moved)\n\t\t\t// so we don't want to include them since the docs say \"first 10 versions\".\n\t\t\tcontinue\n\t\t}\n\n\t\t// Sort the entry into the right bucket:\n\t\tif currDepth < minDepth {\n\t\t\tpart.ShouldPin = append(part.ShouldPin, curr)\n\t\t\tpart.PinSize += nd.Size()\n\t\t} else if currDepth >= minDepth && currDepth < maxDepth {\n\t\t\tpart.QuotaCandidates = append(part.QuotaCandidates, curr)\n\n\t\t\tisPinned, isExplicit, err := fs.pinner.IsNodePinned(nd)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\n\t\t\tif isPinned && !isExplicit {\n\t\t\t\tpart.PinSize += nd.Size()\n\t\t\t}\n\t\t} else {\n\t\t\tpart.DepthCandidates = append(part.DepthCandidates, curr)\n\t\t}\n\n\t\tseen[curr.BackendHash().B58String()] = true\n\t\tcurrDepth++\n\n\t\t// TODO: Optimization: Save depth of last run and abort early if we know\n\t\t// that we unpinned everything at this level already.\n\t}\n\n\tif err := walker.Err(); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn part, nil\n}\n\nfunc (fs *FS) ensurePin(entries []n.ModNode) (uint64, error) {\n\tnewlyPinned := uint64(0)\n\tisPinUnpinned := fs.cfg.Bool(\"repin.pin_unpinned\")\n\n\tfor _, nd := range entries {\n\t\tisPinned, _, err := fs.pinner.IsNodePinned(nd)\n\t\tif err != nil {\n\t\t\treturn newlyPinned, err\n\t\t}\n\t\tif nd.Type() == n.NodeTypeFile && isPinned {\n\t\t\t// let's make sure that this file node is pinned at backend as well\n\t\t\tisCached, err := fs.bk.IsCached(nd.BackendHash())\n\t\t\tif err != nil {\n\t\t\t\treturn newlyPinned, err\n\t\t\t}\n\t\t\tif !isCached {\n\t\t\t\tlog.Warningf(\"The %+v should be cached, but it is not. Recaching\", nd)\n\t\t\t\terr := fs.bk.Pin(nd.BackendHash())\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn newlyPinned, err\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\tif !isPinned && isPinUnpinned {\n\t\t\tif nd.Type() == n.NodeTypeGhost {\n\t\t\t\t// ghosts cannot be pinned\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif err := fs.pinner.PinNode(nd, false); err != nil {\n\t\t\t\treturn newlyPinned, err\n\t\t\t}\n\n\t\t\tnewlyPinned += nd.Size()\n\t\t}\n\t}\n\n\treturn newlyPinned, nil\n}\n\nfunc (fs *FS) ensureUnpin(entries []n.ModNode) (uint64, error) {\n\tsavedStorage := uint64(0)\n\n\tfor _, nd := range entries {\n\t\tisPinned, _, err := fs.pinner.IsNodePinned(nd)\n\t\tif err != nil {\n\t\t\treturn 0, err\n\t\t}\n\n\t\tif isPinned {\n\t\t\texplicit := true // we are unpinning even explicitly pinned\n\t\t\tif err := fs.pinner.UnpinNode(nd, explicit); err != nil {\n\t\t\t\treturn 0, err\n\t\t\t}\n\n\t\t\tsavedStorage += nd.Size()\n\t\t}\n\n\t}\n\n\treturn savedStorage, nil\n}\n\nfunc findLastPinnedIdx(pinner *Pinner, nds []n.ModNode) (int, error) {\n\tfor idx := len(nds) - 1; idx >= 0; idx-- {\n\t\tisPinned, isExplicit, err := pinner.IsNodePinned(nds[idx])\n\t\tif err != nil {\n\t\t\treturn -1, err\n\t\t}\n\n\t\tif isPinned && !isExplicit {\n\t\t\treturn idx, nil\n\t\t}\n\t}\n\n\treturn -1, nil\n}\n\nfunc (fs *FS) balanceQuota(ps []*partition, totalStorage, quota uint64) (uint64, error) {\n\tsort.Slice(ps, func(i, j int) bool {\n\t\treturn ps[i].PinSize < ps[j].PinSize\n\t})\n\n\tidx, empties := 0, 0\n\tsavedStorage := uint64(0)\n\n\t// Try to reduce the pinned storage amount until\n\t// we stay below the determined quota.\n\tfor totalStorage >= quota && empties < len(ps) {\n\t\tcnds := ps[idx%len(ps)].QuotaCandidates\n\t\tif len(cnds) == 0 {\n\t\t\tempties++\n\t\t\tcontinue\n\t\t}\n\n\t\t// Find the last index (i.e. earliest version) that is pinned.\n\t\tlastPinIdx, err := findLastPinnedIdx(fs.pinner, cnds)\n\t\tif err != nil {\n\t\t\treturn 0, err\n\t\t}\n\n\t\tif lastPinIdx < 0 {\n\t\t\tempties++\n\t\t\tps[idx%len(ps)].QuotaCandidates = cnds[:0]\n\t\t\tcontinue\n\t\t}\n\n\t\tcnd := cnds[lastPinIdx]\n\t\ttotalStorage -= cnd.Size()\n\t\tsavedStorage += cnd.Size()\n\n\t\texplicit := true // we are unpinning even explicitly pinned\n\t\tif err := fs.pinner.UnpinNode(cnd, explicit); err != nil {\n\t\t\treturn 0, err\n\t\t}\n\n\t\tps[idx%len(ps)].QuotaCandidates = cnds[:lastPinIdx]\n\t}\n\n\tlog.Infof(\"quota collector unpinned %d bytes\", savedStorage)\n\treturn savedStorage, nil\n}\n\nfunc (fs *FS) repin(root string) error {\n\tfs.mu.Lock()\n\tdefer fs.mu.Unlock()\n\n\t// repinning doesn't modify any metadata,\n\t// but still affects the filesystem.\n\tif fs.readOnly {\n\t\treturn nil\n\t}\n\n\tif !fs.cfg.Bool(\"repin.enabled\") {\n\t\treturn nil\n\t}\n\n\tminDepth := util.Max64(0, fs.cfg.Int(\"repin.min_depth\"))\n\tmaxDepth := util.Max64(1, fs.cfg.Int(\"repin.max_depth\"))\n\tquotaSrc := fs.cfg.String(\"repin.quota\")\n\n\tquota, err := humanize.ParseBytes(quotaSrc)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\trootNd, err := fs.lkr.LookupDirectory(root)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\ttotalStorage := uint64(0)\n\taddedToStorage := uint64(0)\n\tsavedStorage := uint64(0)\n\tparts := []*partition{}\n\n\tlog.Infof(\"repin started (min=%d max=%d quota=%s)\", minDepth, maxDepth, quotaSrc)\n\n\terr = n.Walk(fs.lkr, rootNd, true, func(child n.Node) error {\n\t\tif child.Type() == n.NodeTypeDirectory {\n\t\t\treturn nil\n\t\t}\n\n\t\tmodChild, ok := child.(n.ModNode)\n\t\tif !ok {\n\t\t\treturn e.Wrapf(ie.ErrBadNode, \"repin\")\n\t\t}\n\n\t\tpart, err := fs.partitionNodeHashes(modChild, minDepth, maxDepth)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tpinBytes, err := fs.ensurePin(part.ShouldPin)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tunpinBytes, err := fs.ensureUnpin(part.DepthCandidates)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\ttotalStorage += part.PinSize\n\t\taddedToStorage += pinBytes\n\t\tsavedStorage += unpinBytes\n\n\t\tparts = append(parts, part)\n\t\treturn nil\n\t})\n\n\tif err != nil {\n\t\treturn e.Wrapf(err, \"repin: walk\")\n\t}\n\n\tquotaUnpins, err := fs.balanceQuota(parts, totalStorage, quota)\n\tif err != nil {\n\t\treturn e.Wrapf(err, \"repin: quota balance\")\n\t}\n\n\tsavedStorage += quotaUnpins\n\ttotalStorage -= quotaUnpins\n\n\tif savedStorage >= addedToStorage {\n\t\tlog.Infof(\"repin finished; freed %s, total storage is %s\", humanize.Bytes(savedStorage-addedToStorage), humanize.Bytes(totalStorage))\n\t} else {\n\t\tlog.Infof(\"repin finished; used extra %s, total storage is %s\", humanize.Bytes(addedToStorage-savedStorage), humanize.Bytes(totalStorage))\n\t}\n\treturn nil\n}\n\n// Repin goes over all files in the filesystem and identifies files that need to be unpinned.\n// Only files that are not explicitly pinned, are touched. If a file is explicitly pinned, it will\n// survive the repinning process in any case. The repinning is steered by two config variables:\n//\n// - fs.repin.quota: Maximum amount of pinned storage (excluding explicit pins)\n// - fs.repin.depth: How many versions of a file to keep at least. This trumps quota.\n//\nfunc (fs *FS) Repin(root string) error {\n\tfs.repinControl <- prefixSlash(root)\n\treturn nil\n}\n" }, { "path": "catfs/repin_test.go", "content": "package catfs\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestRepinDepthOnly(t *testing.T) {\n\twithDummyFS(t, func(fs *FS) {\n\t\tfs.cfg.SetBool(\"repin.enabled\", true)\n\t\tfs.cfg.SetString(\"repin.quota\", \"10G\")\n\t\tfs.cfg.SetInt(\"repin.min_depth\", 1)\n\t\tfs.cfg.SetInt(\"repin.max_depth\", 10)\n\n\t\ttestRun(t, fs, 10, 20)\n\t})\n}\n\nfunc TestRepinNoMaxDepth(t *testing.T) {\n\twithDummyFS(t, func(fs *FS) {\n\t\tfs.cfg.SetBool(\"repin.enabled\", true)\n\t\tfs.cfg.SetString(\"repin.quota\", \"10G\")\n\t\tfs.cfg.SetInt(\"repin.min_depth\", 1)\n\t\tfs.cfg.SetInt(\"repin.max_depth\", 100)\n\n\t\ttestRun(t, fs, 20, 20)\n\t})\n}\n\nfunc TestRepinDisabled(t *testing.T) {\n\twithDummyFS(t, func(fs *FS) {\n\t\tfs.cfg.SetBool(\"repin.enabled\", false)\n\t\ttestRun(t, fs, 20, 20)\n\t})\n}\n\nfunc TestRepinQuota(t *testing.T) {\n\twithDummyFS(t, func(fs *FS) {\n\t\tfs.cfg.SetBool(\"repin.enabled\", true)\n\t\tfs.cfg.SetString(\"repin.quota\", \"11B\")\n\t\tfs.cfg.SetInt(\"repin.min_depth\", 1)\n\t\tfs.cfg.SetInt(\"repin.max_depth\", 100)\n\n\t\ttestRun(t, fs, 10, 20)\n\t})\n}\n\nfunc TestRepinKillAll(t *testing.T) {\n\twithDummyFS(t, func(fs *FS) {\n\t\tfs.cfg.SetBool(\"repin.enabled\", true)\n\t\tfs.cfg.SetString(\"repin.quota\", \"0B\")\n\t\tfs.cfg.SetInt(\"repin.min_depth\", 0)\n\t\tfs.cfg.SetInt(\"repin.max_depth\", 0)\n\n\t\ttestRun(t, fs, -1, 20)\n\t})\n}\n\nfunc TestRepinOldBehaviour(t *testing.T) {\n\twithDummyFS(t, func(fs *FS) {\n\t\tfs.cfg.SetBool(\"repin.enabled\", true)\n\t\tfs.cfg.SetString(\"repin.quota\", \"100G\")\n\t\tfs.cfg.SetInt(\"repin.min_depth\", 1)\n\t\tfs.cfg.SetInt(\"repin.max_depth\", 1)\n\n\t\ttestRun(t, fs, 1, 20)\n\t})\n}\n\nfunc testRun(t *testing.T, fs *FS, split, n int) {\n\tfor idx := 0; idx < n; idx++ {\n\t\trequire.Nil(t, fs.Stage(\"/dir/a\", bytes.NewReader([]byte{byte(idx)})))\n\t\trequire.Nil(t, fs.MakeCommit(fmt.Sprintf(\"state: %d\", idx)))\n\t}\n\n\tfor idx := 0; idx < n; idx++ {\n\t\trequire.Nil(t, fs.Pin(\"/dir/a\", \"HEAD\"+strings.Repeat(\"^\", idx), false))\n\t}\n\n\trequire.Nil(t, fs.repin(\"/\"))\n\n\thistA, err := fs.History(\"/dir/a\")\n\trequire.Nil(t, err)\n\n\tfor idx := 0; idx <= split; idx++ {\n\t\trequire.True(t, histA[idx].IsPinned, fmt.Sprintf(\"%d\", idx))\n\t\trequire.False(t, histA[idx].IsExplicit, fmt.Sprintf(\"%d\", idx))\n\t}\n\n\tfor idx := split + 1; idx < n; idx++ {\n\t\trequire.False(t, histA[idx].IsPinned, fmt.Sprintf(\"%d\", idx))\n\t\trequire.False(t, histA[idx].IsExplicit, fmt.Sprintf(\"%d\", idx))\n\t}\n}\n" }, { "path": "catfs/rev.go", "content": "package catfs\n\nimport (\n\t\"fmt\"\n\t\"regexp\"\n\t\"strconv\"\n\t\"strings\"\n\t\"unicode\"\n\n\te \"github.com/pkg/errors\"\n\tc \"github.com/sahib/brig/catfs/core\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n)\n\nvar (\n\tindexCommitPattern = regexp.MustCompile(`^commit\\[([-\\+]{0,1}[0-9]+)\\]$`)\n)\n\n// validateRev check is a rev spec looks like it's valid\n// from a syntactic point of view.\n//\n// A valid ref may contain only letters or numbers, but might end with an\n// arbitrary number of '^' at the end. Unicode is allowed.\n// As special case it might also match indexCommitPattern.\n//\n// If any violation is dected, an error is returned.\nfunc validateRev(rev string) error {\n\tif indexCommitPattern.Match([]byte(rev)) {\n\t\treturn nil\n\t}\n\n\tfoundUp := false\n\tfor _, c := range rev {\n\t\tif unicode.IsLetter(c) || unicode.IsNumber(c) {\n\t\t\tif foundUp {\n\t\t\t\treturn fmt.Errorf(\"normal character after ^\")\n\t\t\t}\n\t\t\tcontinue\n\t\t}\n\n\t\tswitch c {\n\t\tcase '^':\n\t\t\tfoundUp = true\n\t\tdefault:\n\t\t\treturn fmt.Errorf(\"invalid character in ref: `%v`\", c)\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// parseRev resolves a base58 to a commit or if it looks like a refname\n// it tries to resolve that (HEAD, CURR, INIT e.g.).\nfunc parseRev(lkr *c.Linker, rev string) (*n.Commit, error) {\n\tif err := validateRev(rev); err != nil {\n\t\treturn nil, e.Wrapf(err, \"validate\")\n\t}\n\n\tlowerRev := strings.ToLower(rev)\n\tmatches := indexCommitPattern.FindSubmatch([]byte(lowerRev))\n\tif len(matches) >= 2 {\n\t\tindex, err := strconv.ParseInt(string(matches[1]), 10, 64)\n\t\tif err != nil {\n\t\t\treturn nil, e.Wrapf(err, \"failed to parse commit index spec\")\n\t\t}\n\n\t\treturn lkr.CommitByIndex(index)\n\t}\n\n\tpureRev := strings.TrimRight(rev, \"^\")\n\thash, err := lkr.ExpandAbbrev(pureRev)\n\tif err != nil {\n\t\t// Either it was an hash and it is valid,\n\t\t// Or it is a tag name like HEAD (or \"head\")\n\t\tnd, err := lkr.ResolveRef(lowerRev)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tcmt, ok := nd.(*n.Commit)\n\t\tif !ok {\n\t\t\treturn nil, ie.ErrBadNode\n\t\t}\n\n\t\treturn cmt, nil\n\t}\n\n\tactualRev := hash.B58String() + strings.Repeat(\"^\", strings.Count(rev, \"^\"))\n\tnd, err := lkr.ResolveRef(actualRev)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcmt, ok := nd.(*n.Commit)\n\tif !ok {\n\t\treturn nil, ie.ErrBadNode\n\t}\n\n\treturn cmt, nil\n}\n" }, { "path": "catfs/rev_test.go", "content": "package catfs\n\nimport (\n\t\"testing\"\n\n\tc \"github.com/sahib/brig/catfs/core\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestRevParse(t *testing.T) {\n\tc.WithDummyLinker(t, func(lkr *c.Linker) {\n\t\tinit, err := parseRev(lkr, \"commit[0]\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, \"init\", init.Message())\n\t})\n}\n" }, { "path": "catfs/vcs/capnp/patch.capnp", "content": "using Go = import \"/go.capnp\";\nusing Nodes = import \"../../nodes/capnp/nodes.capnp\";\n\n@0xb943b54bf1683782;\n\n$Go.package(\"capnp\");\n$Go.import(\"github.com/sahib/brig/catfs/vcs/capnp\");\n\nstruct Change $Go.doc(\"Change describes a single change\") {\n mask @0 :UInt64;\n head @1 :Nodes.Node;\n next @2 :Nodes.Node;\n curr @3 :Nodes.Node;\n movedTo @4 :Text;\n wasPreviouslyAt @5 :Text;\n}\n\nstruct Patch $Go.doc(\"Patch contains a single change\") {\n fromIndex @0 :Int64;\n currIndex @1 :Int64;\n changes @2 :List(Change);\n}\n\nstruct Patches $Go.doc(\"Patches contains several patches\") {\n patches @0 :List(Patch);\n}\n" }, { "path": "catfs/vcs/capnp/patch.capnp.go", "content": "// Code generated by capnpc-go. DO NOT EDIT.\n\npackage capnp\n\nimport (\n\tcapnp2 \"github.com/sahib/brig/catfs/nodes/capnp\"\n\tcapnp \"zombiezen.com/go/capnproto2\"\n\ttext \"zombiezen.com/go/capnproto2/encoding/text\"\n\tschemas \"zombiezen.com/go/capnproto2/schemas\"\n)\n\n// Change describes a single change\ntype Change struct{ capnp.Struct }\n\n// Change_TypeID is the unique identifier for the type Change.\nconst Change_TypeID = 0x9592300df48789af\n\nfunc NewChange(s *capnp.Segment) (Change, error) {\n\tst, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 8, PointerCount: 5})\n\treturn Change{st}, err\n}\n\nfunc NewRootChange(s *capnp.Segment) (Change, error) {\n\tst, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 8, PointerCount: 5})\n\treturn Change{st}, err\n}\n\nfunc ReadRootChange(msg *capnp.Message) (Change, error) {\n\troot, err := msg.RootPtr()\n\treturn Change{root.Struct()}, err\n}\n\nfunc (s Change) String() string {\n\tstr, _ := text.Marshal(0x9592300df48789af, s.Struct)\n\treturn str\n}\n\nfunc (s Change) Mask() uint64 {\n\treturn s.Struct.Uint64(0)\n}\n\nfunc (s Change) SetMask(v uint64) {\n\ts.Struct.SetUint64(0, v)\n}\n\nfunc (s Change) Head() (capnp2.Node, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn capnp2.Node{Struct: p.Struct()}, err\n}\n\nfunc (s Change) HasHead() bool {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Change) SetHead(v capnp2.Node) error {\n\treturn s.Struct.SetPtr(0, v.Struct.ToPtr())\n}\n\n// NewHead sets the head field to a newly\n// allocated capnp2.Node struct, preferring placement in s's segment.\nfunc (s Change) NewHead() (capnp2.Node, error) {\n\tss, err := capnp2.NewNode(s.Struct.Segment())\n\tif err != nil {\n\t\treturn capnp2.Node{}, err\n\t}\n\terr = s.Struct.SetPtr(0, ss.Struct.ToPtr())\n\treturn ss, err\n}\n\nfunc (s Change) Next() (capnp2.Node, error) {\n\tp, err := s.Struct.Ptr(1)\n\treturn capnp2.Node{Struct: p.Struct()}, err\n}\n\nfunc (s Change) HasNext() bool {\n\tp, err := s.Struct.Ptr(1)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Change) SetNext(v capnp2.Node) error {\n\treturn s.Struct.SetPtr(1, v.Struct.ToPtr())\n}\n\n// NewNext sets the next field to a newly\n// allocated capnp2.Node struct, preferring placement in s's segment.\nfunc (s Change) NewNext() (capnp2.Node, error) {\n\tss, err := capnp2.NewNode(s.Struct.Segment())\n\tif err != nil {\n\t\treturn capnp2.Node{}, err\n\t}\n\terr = s.Struct.SetPtr(1, ss.Struct.ToPtr())\n\treturn ss, err\n}\n\nfunc (s Change) Curr() (capnp2.Node, error) {\n\tp, err := s.Struct.Ptr(2)\n\treturn capnp2.Node{Struct: p.Struct()}, err\n}\n\nfunc (s Change) HasCurr() bool {\n\tp, err := s.Struct.Ptr(2)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Change) SetCurr(v capnp2.Node) error {\n\treturn s.Struct.SetPtr(2, v.Struct.ToPtr())\n}\n\n// NewCurr sets the curr field to a newly\n// allocated capnp2.Node struct, preferring placement in s's segment.\nfunc (s Change) NewCurr() (capnp2.Node, error) {\n\tss, err := capnp2.NewNode(s.Struct.Segment())\n\tif err != nil {\n\t\treturn capnp2.Node{}, err\n\t}\n\terr = s.Struct.SetPtr(2, ss.Struct.ToPtr())\n\treturn ss, err\n}\n\nfunc (s Change) MovedTo() (string, error) {\n\tp, err := s.Struct.Ptr(3)\n\treturn p.Text(), err\n}\n\nfunc (s Change) HasMovedTo() bool {\n\tp, err := s.Struct.Ptr(3)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Change) MovedToBytes() ([]byte, error) {\n\tp, err := s.Struct.Ptr(3)\n\treturn p.TextBytes(), err\n}\n\nfunc (s Change) SetMovedTo(v string) error {\n\treturn s.Struct.SetText(3, v)\n}\n\nfunc (s Change) WasPreviouslyAt() (string, error) {\n\tp, err := s.Struct.Ptr(4)\n\treturn p.Text(), err\n}\n\nfunc (s Change) HasWasPreviouslyAt() bool {\n\tp, err := s.Struct.Ptr(4)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Change) WasPreviouslyAtBytes() ([]byte, error) {\n\tp, err := s.Struct.Ptr(4)\n\treturn p.TextBytes(), err\n}\n\nfunc (s Change) SetWasPreviouslyAt(v string) error {\n\treturn s.Struct.SetText(4, v)\n}\n\n// Change_List is a list of Change.\ntype Change_List struct{ capnp.List }\n\n// NewChange creates a new list of Change.\nfunc NewChange_List(s *capnp.Segment, sz int32) (Change_List, error) {\n\tl, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 8, PointerCount: 5}, sz)\n\treturn Change_List{l}, err\n}\n\nfunc (s Change_List) At(i int) Change { return Change{s.List.Struct(i)} }\n\nfunc (s Change_List) Set(i int, v Change) error { return s.List.SetStruct(i, v.Struct) }\n\nfunc (s Change_List) String() string {\n\tstr, _ := text.MarshalList(0x9592300df48789af, s.List)\n\treturn str\n}\n\n// Change_Promise is a wrapper for a Change promised by a client call.\ntype Change_Promise struct{ *capnp.Pipeline }\n\nfunc (p Change_Promise) Struct() (Change, error) {\n\ts, err := p.Pipeline.Struct()\n\treturn Change{s}, err\n}\n\nfunc (p Change_Promise) Head() capnp2.Node_Promise {\n\treturn capnp2.Node_Promise{Pipeline: p.Pipeline.GetPipeline(0)}\n}\n\nfunc (p Change_Promise) Next() capnp2.Node_Promise {\n\treturn capnp2.Node_Promise{Pipeline: p.Pipeline.GetPipeline(1)}\n}\n\nfunc (p Change_Promise) Curr() capnp2.Node_Promise {\n\treturn capnp2.Node_Promise{Pipeline: p.Pipeline.GetPipeline(2)}\n}\n\n// Patch contains a single change\ntype Patch struct{ capnp.Struct }\n\n// Patch_TypeID is the unique identifier for the type Patch.\nconst Patch_TypeID = 0x927c7336e3054805\n\nfunc NewPatch(s *capnp.Segment) (Patch, error) {\n\tst, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 16, PointerCount: 1})\n\treturn Patch{st}, err\n}\n\nfunc NewRootPatch(s *capnp.Segment) (Patch, error) {\n\tst, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 16, PointerCount: 1})\n\treturn Patch{st}, err\n}\n\nfunc ReadRootPatch(msg *capnp.Message) (Patch, error) {\n\troot, err := msg.RootPtr()\n\treturn Patch{root.Struct()}, err\n}\n\nfunc (s Patch) String() string {\n\tstr, _ := text.Marshal(0x927c7336e3054805, s.Struct)\n\treturn str\n}\n\nfunc (s Patch) FromIndex() int64 {\n\treturn int64(s.Struct.Uint64(0))\n}\n\nfunc (s Patch) SetFromIndex(v int64) {\n\ts.Struct.SetUint64(0, uint64(v))\n}\n\nfunc (s Patch) CurrIndex() int64 {\n\treturn int64(s.Struct.Uint64(8))\n}\n\nfunc (s Patch) SetCurrIndex(v int64) {\n\ts.Struct.SetUint64(8, uint64(v))\n}\n\nfunc (s Patch) Changes() (Change_List, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn Change_List{List: p.List()}, err\n}\n\nfunc (s Patch) HasChanges() bool {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Patch) SetChanges(v Change_List) error {\n\treturn s.Struct.SetPtr(0, v.List.ToPtr())\n}\n\n// NewChanges sets the changes field to a newly\n// allocated Change_List, preferring placement in s's segment.\nfunc (s Patch) NewChanges(n int32) (Change_List, error) {\n\tl, err := NewChange_List(s.Struct.Segment(), n)\n\tif err != nil {\n\t\treturn Change_List{}, err\n\t}\n\terr = s.Struct.SetPtr(0, l.List.ToPtr())\n\treturn l, err\n}\n\n// Patch_List is a list of Patch.\ntype Patch_List struct{ capnp.List }\n\n// NewPatch creates a new list of Patch.\nfunc NewPatch_List(s *capnp.Segment, sz int32) (Patch_List, error) {\n\tl, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 16, PointerCount: 1}, sz)\n\treturn Patch_List{l}, err\n}\n\nfunc (s Patch_List) At(i int) Patch { return Patch{s.List.Struct(i)} }\n\nfunc (s Patch_List) Set(i int, v Patch) error { return s.List.SetStruct(i, v.Struct) }\n\nfunc (s Patch_List) String() string {\n\tstr, _ := text.MarshalList(0x927c7336e3054805, s.List)\n\treturn str\n}\n\n// Patch_Promise is a wrapper for a Patch promised by a client call.\ntype Patch_Promise struct{ *capnp.Pipeline }\n\nfunc (p Patch_Promise) Struct() (Patch, error) {\n\ts, err := p.Pipeline.Struct()\n\treturn Patch{s}, err\n}\n\n// Patches contains several patches\ntype Patches struct{ capnp.Struct }\n\n// Patches_TypeID is the unique identifier for the type Patches.\nconst Patches_TypeID = 0xc2984f083ea5351d\n\nfunc NewPatches(s *capnp.Segment) (Patches, error) {\n\tst, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 1})\n\treturn Patches{st}, err\n}\n\nfunc NewRootPatches(s *capnp.Segment) (Patches, error) {\n\tst, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 1})\n\treturn Patches{st}, err\n}\n\nfunc ReadRootPatches(msg *capnp.Message) (Patches, error) {\n\troot, err := msg.RootPtr()\n\treturn Patches{root.Struct()}, err\n}\n\nfunc (s Patches) String() string {\n\tstr, _ := text.Marshal(0xc2984f083ea5351d, s.Struct)\n\treturn str\n}\n\nfunc (s Patches) Patches() (Patch_List, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn Patch_List{List: p.List()}, err\n}\n\nfunc (s Patches) HasPatches() bool {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Patches) SetPatches(v Patch_List) error {\n\treturn s.Struct.SetPtr(0, v.List.ToPtr())\n}\n\n// NewPatches sets the patches field to a newly\n// allocated Patch_List, preferring placement in s's segment.\nfunc (s Patches) NewPatches(n int32) (Patch_List, error) {\n\tl, err := NewPatch_List(s.Struct.Segment(), n)\n\tif err != nil {\n\t\treturn Patch_List{}, err\n\t}\n\terr = s.Struct.SetPtr(0, l.List.ToPtr())\n\treturn l, err\n}\n\n// Patches_List is a list of Patches.\ntype Patches_List struct{ capnp.List }\n\n// NewPatches creates a new list of Patches.\nfunc NewPatches_List(s *capnp.Segment, sz int32) (Patches_List, error) {\n\tl, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 0, PointerCount: 1}, sz)\n\treturn Patches_List{l}, err\n}\n\nfunc (s Patches_List) At(i int) Patches { return Patches{s.List.Struct(i)} }\n\nfunc (s Patches_List) Set(i int, v Patches) error { return s.List.SetStruct(i, v.Struct) }\n\nfunc (s Patches_List) String() string {\n\tstr, _ := text.MarshalList(0xc2984f083ea5351d, s.List)\n\treturn str\n}\n\n// Patches_Promise is a wrapper for a Patches promised by a client call.\ntype Patches_Promise struct{ *capnp.Pipeline }\n\nfunc (p Patches_Promise) Struct() (Patches, error) {\n\ts, err := p.Pipeline.Struct()\n\treturn Patches{s}, err\n}\n\nconst schema_b943b54bf1683782 = \"x\\xda\\x84\\x93\\xc1k\\x13O\\x1c\\xc5\\xbf\\xef;\\xbb\\xbf\\xb4\" +\n\t\"?R\\xd319\\x88T\\xba\\x88\\x97z\\xb0-E\\x05\\x0f\" +\n\t\"\\xd6Z\\x05E\\xc5\\x8c(\\xeaEX7\\xd3n0\\xd9\\x84\" +\n\t\"\\xcc6V\\xb0\\x04z\\x11=V\\x85\\x16\\x14+\\xb4b\\xa1\" +\n\t\"R/\\x82\\x1e\\xbc\\x08\\xfe\\x0b\\xfe\\x03=\\x89'\\xc1K{\" +\n\t\"Y\\x99$\\xa6\\xa5\\x16{[>\\xf3\\xd8\\xd9\\xcf\\xe3\\xed\\x10\" +\n\t\"p\\x86\\x87\\xdd\\x0c\\x88T\\xda\\xfd/q/\\xb8\\xeb'\\xcc\" +\n\t\"\\xc39R}\\xe0d\\xf6d\\xf8\\xf3\\xd2\\x87\\xf1O\\xe4\\\"\" +\n\t\"E4\\\\\\xde\\x0f9\\x93\\x923\\xfdre\\x8d\\x90\\xac=\" +\n\t\"~\\xf4\\xabgh\\xee\\xb9\\xcdb[\\xd6M\\x11\\x8d\\xdc\\xc6\" +\n\t\"Ad\\x8bHe\\x8b\\xe8\\x1fY\\xc0M\\x10\\x92C\\xc7\\x97\" +\n\t\"Ow]\\x9d\\xffB\\xb2\\x0f;\\xde=\\xf2\\x9d\\x0f#\\xbb\" +\n\t\"\\xc1\\xa9\\xec\\x06\\xf7g\\x07\\xc4(!\\x09\\xfcx\\xc2\\x0c\\xd6\" +\n\t\"\\x03a\\x06\\x03\\xbf\\x1aU\\x07\\xab~\\x1c\\x84\\xc7\\x9a\\xcf\\xa7\" +\n\t\"\\xf2~\\x1c \\xcc\\x03\\xca\\x01'w\\x9e\\xbeR\\x9f\\xbf=\" +\n\t\"\\xf9J\\xcaa\\x8c\\xf5\\x01i\\\"\\x89\\xcd\\xc4\\xa6B/\\xa8\" +\n\t\"p\\x14\\xfb\\xc5\\xc8x\\xbeg\\x8a\\xd1dI{\\xa3A\\xe8\" +\n\t\"G\\x93\\xdaZ\\x0b\\x87\\xc8\\x01\\x91<\\x7f\\x8dH\\x9d\\x13P\" +\n\t\"y\\x86\\x04r\\xb0\\xf0\\x8a\\x85\\x97\\x05\\xd4-\\x068\\x07&\" +\n\t\"\\x927\\xce\\x12\\xa9\\xbc\\x80*1\\x92\\x89Z\\xa5|1*\" +\n\t\"h\\xc24\\\\b\\xb8\\xf6\\xcb\\xa7j\\xb5\\x1d\\xac\\xd1\\xba\\xd0\" +\n\t\"`\\x1f!/\\x80\\xde\\xad\\xfa\\x08\\x16\\xfe[w<\\xf4#\" +\n\t\"1\\xa9w\\xf7\\xf5\\x9a\\xbe\\xc3\\xf8\\x1f\\xc9x\\xf3\\x16\\xaf \" +\n\t\"\\xb4\\x09j\\xc5\\xbbz\\x9br\\xdb\\x18\\xea@\\xc7x\\xe1(\" +\n\t\"\\x91z&\\xa0\\x16\\x19\\x7f\\x84_Z6/\\xa0\\x96\\x18\\x92\" +\n\t\"\\xd12~m\\xe1\\x0b\\x01\\xf5\\x96!\\x05\\xe7 \\x88\\xe4\\xb2\" +\n\t\"\\x85\\x8b\\x02j\\x95!\\x1d\\x91\\x83C$Wl7K\\x02\" +\n\t\"\\xea=C\\xbaN\\x0e.\\x91|7K\\xa4V\\x05\\xd4G\" +\n\t\"F\\xa6\\xec\\x9b{\\xe8&F7!\\x13j\\xbf\\x80\\xded\" +\n\t\"}s\\xa2\\xda\\xf81\\xf0\\xc6v\\xd1K\\xc8Dz:\\xde\" +\n\t\"\\x05\\xdb^\\xff\\xc6\\x8dr\\xa5\\xae\\x0b\\xd7+H\\x13#M\" +\n\t\"H\\xee\\xfb&_\\xd3\\xf5\\\"*S\\xa6\\xf4`,\\xa6\\xce\" +\n\t\"\\xc9\\x9e\\x93J\\x85\\xda\\xecYrsU\\xdax\\\"\\xa8\\xb4\" +\n\t\"wet]\\xd7\\xfc\\x92Wm\\x9d\\x10\\x94\\xd3)\\xb9\\xc7\" +\n\t\"6\\xd2%\\xa0\\x8e0\\x1a\\xed\\xc0\\xd6\\x0a:?\\\\k\\x05\" +\n\t\"\\xbf\\x03\\x00\\x00\\xff\\xff\\xe5\\xc7\\xe9\\x1d\"\n\nfunc init() {\n\tschemas.Register(schema_b943b54bf1683782,\n\t\t0x927c7336e3054805,\n\t\t0x9592300df48789af,\n\t\t0xc2984f083ea5351d)\n}\n" }, { "path": "catfs/vcs/change.go", "content": "package vcs\n\nimport (\n\t\"fmt\"\n\t\"path\"\n\t\"strings\"\n\n\te \"github.com/pkg/errors\"\n\tc \"github.com/sahib/brig/catfs/core\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\tcapnp_model \"github.com/sahib/brig/catfs/nodes/capnp\"\n\tcapnp_patch \"github.com/sahib/brig/catfs/vcs/capnp\"\n\tlog \"github.com/sirupsen/logrus\"\n\tcapnp \"zombiezen.com/go/capnproto2\"\n)\n\nconst (\n\t// ChangeTypeNone means that a node did not change (compared to HEAD)\n\tChangeTypeNone = ChangeType(0)\n\t// ChangeTypeAdd says that the node was initially added after HEAD.\n\tChangeTypeAdd = ChangeType(1 << iota)\n\t// ChangeTypeModify says that the the node was modified after HEAD\n\tChangeTypeModify\n\t// ChangeTypeMove says that the node was moved after HEAD.\n\t// Note that Move and Modify may happen at the same time.\n\tChangeTypeMove\n\t// ChangeTypeRemove says that the node was removed after HEAD.\n\tChangeTypeRemove\n)\n\n// ChangeType is a mask of possible state change events.\ntype ChangeType uint8\n\n// String will convert a ChangeType to a human readable form\nfunc (ct ChangeType) String() string {\n\tv := []string{}\n\n\tif ct&ChangeTypeAdd != 0 {\n\t\tv = append(v, \"added\")\n\t}\n\tif ct&ChangeTypeModify != 0 {\n\t\tv = append(v, \"modified\")\n\t}\n\tif ct&ChangeTypeMove != 0 {\n\t\tv = append(v, \"moved\")\n\t}\n\tif ct&ChangeTypeRemove != 0 {\n\t\tv = append(v, \"removed\")\n\t}\n\n\tif len(v) == 0 {\n\t\treturn \"none\"\n\t}\n\n\treturn strings.Join(v, \"|\")\n}\n\n// IsCompatible checks if two change masks are compatible.\n// Changes are compatible when they can be both applied\n// without loosing any content. We may loose metadata though,\n// e.g. when one side was moved, but the other removed:\n// Here the remove would win and no move is counted.\nfunc (ct ChangeType) IsCompatible(ot ChangeType) bool {\n\tmodifyMask := ChangeTypeAdd | ChangeTypeModify\n\treturn ct&modifyMask == 0 || ot&modifyMask == 0\n}\n\n///////////////////////////\n\n// Change represents a single change of a node between two commits.\ntype Change struct {\n\t// Mask is a bitmask of changes that were made.\n\t// It describes the change that was made between `Next` to `Head`\n\t// and which is part of `Head`.\n\tMask ChangeType\n\n\t// Head is the commit that was the current HEAD when this change happened.\n\t// Note that this is NOT the commit that contains the change, but the commit before.\n\tHead *n.Commit\n\n\t// Next is the commit that comes before `Head`.\n\tNext *n.Commit\n\n\t// Curr is the node with the attributes at a specific state\n\tCurr n.ModNode\n\n\t// MovedTo is only filled for ghosts that were the source\n\t// of a move. It's the path of the node it was moved to.\n\tMovedTo string\n\n\t// WasPreviouslyAt points to the place `Curr` was at\n\t// before a move. On changes without a move this is empty.\n\tWasPreviouslyAt string\n}\n\nfunc (ch *Change) String() string {\n\tmovedTo := \"\"\n\tif len(ch.MovedTo) != 0 {\n\t\tmovedTo = fmt.Sprintf(\" (now %s)\", ch.MovedTo)\n\t}\n\n\tprevAt := \"\"\n\tif len(ch.WasPreviouslyAt) != 0 {\n\t\tprevAt = fmt.Sprintf(\" (was %s)\", ch.WasPreviouslyAt)\n\t}\n\n\treturn fmt.Sprintf(\"<%s:%s%s%s>\", ch.Curr.Path(), ch.Mask, prevAt, movedTo)\n}\n\nfunc replayAddWithUnpacking(lkr *c.Linker, ch *Change) error {\n\t// If it's an ghost, unpack it first: It will be added as if it was\n\t// never a ghost, but since the change mask has the\n\t// ChangeTypeRemove flag set, it will removed directly after.\n\tcurrNd := ch.Curr\n\tif ch.Curr.Type() == n.NodeTypeGhost {\n\t\tcurrGhost, ok := ch.Curr.(*n.Ghost)\n\t\tif !ok {\n\t\t\treturn ie.ErrBadNode\n\t\t}\n\n\t\tcurrNd = currGhost.OldNode()\n\t}\n\n\t// Check the type of the old node:\n\toldNd, err := lkr.LookupModNode(currNd.Path())\n\tif err != nil && !ie.IsNoSuchFileError(err) {\n\t\treturn err\n\t}\n\n\t// If the types are conflicting we have to remove the existing node.\n\tif oldNd != nil && oldNd.Type() != currNd.Type() {\n\t\tif oldNd.Type() == n.NodeTypeGhost {\n\t\t\t// the oldNd node is already deleted, no need to do anything special\n\t\t\treturn replayAdd(lkr, currNd)\n\t\t}\n\t\t_, _, err := c.Remove(lkr, oldNd, true, true)\n\t\tif err != nil {\n\t\t\treturn e.Wrapf(err, \"replay: type-conflict-remove\")\n\t\t}\n\t}\n\n\treturn replayAdd(lkr, currNd)\n}\n\nfunc replayAdd(lkr *c.Linker, currNd n.ModNode) error {\n\tswitch currNd.(type) {\n\tcase *n.File:\n\t\tif _, err := c.Mkdir(lkr, path.Dir(currNd.Path()), true); err != nil {\n\t\t\treturn e.Wrapf(err, \"replay: mkdir\")\n\t\t}\n\n\t\tif _, err := c.StageFromFileNode(lkr, currNd.(*n.File)); err != nil {\n\t\t\treturn e.Wrapf(err, \"replay: stage\")\n\t\t}\n\tcase *n.Directory:\n\t\tif _, err := c.Mkdir(lkr, currNd.Path(), true); err != nil {\n\t\t\treturn e.Wrapf(err, \"replay: mkdir\")\n\t\t}\n\tdefault:\n\t\treturn e.Wrapf(ie.ErrBadNode, \"replay: modify\")\n\t}\n\n\treturn nil\n}\n\nfunc replayMove(lkr *c.Linker, ch *Change) error {\n\tif ch.MovedTo != \"\" {\n\t\toldNd, err := lkr.LookupModNode(ch.Curr.Path())\n\t\tif err != nil && !ie.IsNoSuchFileError(err) {\n\t\t\treturn err\n\t\t}\n\n\t\tif _, err := c.Mkdir(lkr, path.Dir(ch.MovedTo), true); err != nil {\n\t\t\treturn e.Wrapf(err, \"replay: mkdir\")\n\t\t}\n\n\t\tif oldNd != nil {\n\t\t\tif err := c.Move(lkr, oldNd, ch.MovedTo); err != nil {\n\t\t\t\treturn e.Wrapf(err, \"replay: move\")\n\t\t\t}\n\t\t}\n\t}\n\n\tif ch.Curr.Type() != n.NodeTypeGhost {\n\t\tif _, err := lkr.LookupModNode(ch.Curr.Path()); ie.IsNoSuchFileError(err) {\n\t\t\tif err := replayAdd(lkr, ch.Curr); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t}\n\n\tif ch.WasPreviouslyAt != \"\" {\n\t\toldNd, err := lkr.LookupModNode(ch.WasPreviouslyAt)\n\t\tif err != nil && !ie.IsNoSuchFileError(err) {\n\t\t\treturn err\n\t\t}\n\n\t\tif oldNd != nil {\n\t\t\tif oldNd.Type() != n.NodeTypeGhost {\n\t\t\t\tif _, _, err := c.Remove(lkr, oldNd, true, true); err != nil {\n\t\t\t\t\treturn e.Wrap(err, \"replay: move: remove old\")\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\tif err := replayAddMoveMapping(lkr, ch.WasPreviouslyAt, ch.Curr.Path()); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc replayAddMoveMapping(lkr *c.Linker, oldPath, newPath string) error {\n\tnewNd, err := lkr.LookupModNode(newPath)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\toldNd, err := lkr.LookupModNode(oldPath)\n\tif err != nil && !ie.IsNoSuchFileError(err) {\n\t\treturn nil\n\t}\n\n\tif oldNd == nil {\n\t\treturn nil\n\t}\n\n\tlog.Debugf(\"adding move mapping: %s %s\", oldPath, newPath)\n\treturn lkr.AddMoveMapping(oldNd.Inode(), newNd.Inode())\n}\n\nfunc replayRemove(lkr *c.Linker, ch *Change) error {\n\tcurrNd, err := lkr.LookupModNode(ch.Curr.Path())\n\tif err != nil {\n\t\treturn e.Wrapf(err, \"replay: lookup: %v\", ch.Curr.Path())\n\t}\n\n\tif currNd.Type() != n.NodeTypeGhost {\n\t\tif _, _, err := c.Remove(lkr, currNd, true, true); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// Replay applies the change `ch` onto `lkr` by redoing the same operations:\n// move, remove, modify, add. Commits are not replayed, everything happens in\n// lkr.Status() without creating a new commit.\nfunc (ch *Change) Replay(lkr *c.Linker) error {\n\treturn lkr.Atomic(func() (bool, error) {\n\t\tif ch.Mask&(ChangeTypeModify|ChangeTypeAdd) != 0 {\n\t\t\t// Something needs to be done based on the type.\n\t\t\t// Either create/update a new file or create a directory.\n\t\t\tif err := replayAddWithUnpacking(lkr, ch); err != nil {\n\t\t\t\treturn true, err\n\t\t\t}\n\t\t}\n\n\t\tif ch.Mask&ChangeTypeMove != 0 {\n\t\t\tif err := replayMove(lkr, ch); err != nil {\n\t\t\t\treturn true, err\n\t\t\t}\n\t\t}\n\n\t\t// We should only remove a node if we're getting a ghost in ch.Curr.\n\t\t// Otherwise the node might have been removed and added again.\n\t\tif ch.Mask&ChangeTypeRemove != 0 && ch.Curr.Type() == n.NodeTypeGhost {\n\t\t\tif err := replayRemove(lkr, ch); err != nil {\n\t\t\t\treturn true, err\n\t\t\t}\n\t\t}\n\n\t\treturn false, nil\n\t})\n}\n\nfunc (ch *Change) toCapnpChange(seg *capnp.Segment, capCh *capnp_patch.Change) error {\n\tcapCurrNd, err := capnp_model.NewNode(seg)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif err := ch.Curr.ToCapnpNode(seg, capCurrNd); err != nil {\n\t\treturn err\n\t}\n\n\tcapHeadNd, err := capnp_model.NewNode(seg)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif err := ch.Head.ToCapnpNode(seg, capHeadNd); err != nil {\n\t\treturn err\n\t}\n\n\tcapNextNd, err := capnp_model.NewNode(seg)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif err := ch.Next.ToCapnpNode(seg, capNextNd); err != nil {\n\t\treturn err\n\t}\n\n\tif err := capCh.SetCurr(capCurrNd); err != nil {\n\t\treturn err\n\t}\n\n\tif err := capCh.SetHead(capHeadNd); err != nil {\n\t\treturn err\n\t}\n\n\tif err := capCh.SetNext(capNextNd); err != nil {\n\t\treturn err\n\t}\n\n\tif err := capCh.SetMovedTo(ch.MovedTo); err != nil {\n\t\treturn err\n\t}\n\n\tif err := capCh.SetWasPreviouslyAt(ch.WasPreviouslyAt); err != nil {\n\t\treturn err\n\t}\n\n\tcapCh.SetMask(uint64(ch.Mask))\n\treturn nil\n\n}\n\n// ToCapnp converts a change to a capnproto message.\nfunc (ch *Change) ToCapnp() (*capnp.Message, error) {\n\tmsg, seg, err := capnp.NewMessage(capnp.SingleSegment(nil))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapCh, err := capnp_patch.NewRootChange(seg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := ch.toCapnpChange(seg, &capCh); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn msg, nil\n}\n\nfunc (ch *Change) fromCapnpChange(capCh capnp_patch.Change) error {\n\tcapHeadNd, err := capCh.Head()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tch.Head = &n.Commit{}\n\tif err := ch.Head.FromCapnpNode(capHeadNd); err != nil {\n\t\treturn err\n\t}\n\n\tcapNextNd, err := capCh.Next()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tch.Next = &n.Commit{}\n\tif err := ch.Next.FromCapnpNode(capNextNd); err != nil {\n\t\treturn err\n\t}\n\n\tcapCurrNd, err := capCh.Curr()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tcurrNd, err := n.CapNodeToNode(capCurrNd)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tcurrModNd, ok := currNd.(n.ModNode)\n\tif !ok {\n\t\treturn e.Wrapf(ie.ErrBadNode, \"unmarshalled node is no mod node\")\n\t}\n\n\tch.Curr = currModNd\n\n\tmovedTo, err := capCh.MovedTo()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\twasPreviouslyAt, err := capCh.WasPreviouslyAt()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tch.MovedTo = movedTo\n\tch.WasPreviouslyAt = wasPreviouslyAt\n\tch.Mask = ChangeType(capCh.Mask())\n\treturn nil\n}\n\n// FromCapnp deserializes `msg` and writes it to `ch`.\nfunc (ch *Change) FromCapnp(msg *capnp.Message) error {\n\tcapCh, err := capnp_patch.ReadRootChange(msg)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn ch.fromCapnpChange(capCh)\n}\n\n// CombineChanges compresses a list of changes (in a lossy way) to one Change.\n// The one change should be enough to re-create the changes that were made.\nfunc CombineChanges(changes []*Change) *Change {\n\tif len(changes) == 0 {\n\t\treturn nil\n\t}\n\n\t// Only take the latest changes:\n\tch := &Change{\n\t\tMask: ChangeType(0),\n\t\tHead: changes[0].Head,\n\t\tNext: changes[0].Next,\n\t\tCurr: changes[0].Curr,\n\t}\n\n\t// If the node moved, save the original path in MovedTo:\n\tpathChanged := changes[0].Curr.Path() != changes[len(changes)-1].Curr.Path()\n\tisGhost := changes[0].Curr.Type() == n.NodeTypeGhost\n\n\t// Combine the mask:\n\tfor _, change := range changes {\n\t\tch.Mask |= change.Mask\n\t}\n\n\tif ch.Mask&ChangeTypeMove != 0 {\n\t\tfor idx := len(changes) - 1; idx >= 0; idx-- {\n\t\t\tif refPath := changes[idx].MovedTo; refPath != \"\" {\n\t\t\t\tch.MovedTo = refPath\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\n\t\tfor idx := len(changes) - 1; idx >= 0; idx-- {\n\t\t\tif refPath := changes[idx].WasPreviouslyAt; refPath != \"\" {\n\t\t\t\tch.WasPreviouslyAt = refPath\n\t\t\t\tpathChanged = refPath != changes[0].Curr.Path()\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t}\n\n\t// If the path did not really change, we do not want to have ChangeTypeMove\n\t// in the mask. This is to protect against circular moves. If it's a ghost\n\t// we should still include it though (for WasPreviouslyAt)\n\tif !pathChanged && !isGhost {\n\t\tch.Mask &= ^ChangeTypeMove\n\n\t}\n\n\t// If the last change was not a remove, we do not need to\n\tif changes[0].Mask&ChangeTypeRemove == 0 && !isGhost {\n\t\tch.Mask &= ^ChangeTypeRemove\n\t}\n\n\treturn ch\n}\n" }, { "path": "catfs/vcs/change_test.go", "content": "package vcs\n\nimport (\n\t\"testing\"\n\n\tc \"github.com/sahib/brig/catfs/core\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestChangeMarshalling(t *testing.T) {\n\tc.WithDummyLinker(t, func(lkr *c.Linker) {\n\t\thead, err := lkr.Head()\n\t\trequire.Nil(t, err)\n\n\t\tcurr := c.MustTouch(t, lkr, \"/x\", 1)\n\t\tnext := c.MustCommit(t, lkr, \"hello\")\n\n\t\tchange := &Change{\n\t\t\tMask: ChangeTypeMove | ChangeTypeRemove,\n\t\t\tHead: head,\n\t\t\tNext: next,\n\t\t\tCurr: curr,\n\t\t\tMovedTo: \"/something\",\n\t\t}\n\n\t\tmsg, err := change.ToCapnp()\n\t\trequire.Nil(t, err)\n\n\t\tnewChange := &Change{}\n\t\trequire.Nil(t, newChange.FromCapnp(msg))\n\n\t\trequire.Equal(t, newChange.MovedTo, \"/something\")\n\t\trequire.Equal(t, newChange.Mask, ChangeTypeMove|ChangeTypeRemove)\n\t\trequire.Equal(t, newChange.Curr, curr)\n\t\trequire.Equal(t, newChange.Head, head)\n\t\trequire.Equal(t, newChange.Next, next)\n\n\t\t// This check helps failing when adding new fields:\n\t\trequire.Equal(t, change, newChange)\n\t})\n}\n\nfunc TestChangeCombine(t *testing.T) {\n\tc.WithDummyLinker(t, func(lkr *c.Linker) {\n\t\tx := c.MustTouch(t, lkr, \"/x\", 1)\n\t\tc.MustCommit(t, lkr, \"1\")\n\t\tc.MustModify(t, lkr, x, 2)\n\t\tc.MustCommit(t, lkr, \"2\")\n\t\ty := c.MustMove(t, lkr, x, \"/y\")\n\t\tc.MustCommit(t, lkr, \"move\")\n\t\tc.MustRemove(t, lkr, y)\n\t\tghost, err := lkr.LookupGhost(\"/y\")\n\t\trequire.Nil(t, err)\n\n\t\tstatus, err := lkr.Status()\n\t\trequire.Nil(t, err)\n\n\t\tchanges, err := History(lkr, ghost, status, nil)\n\t\trequire.Nil(t, err)\n\t\trequire.Len(t, changes, 4)\n\t\trequire.Equal(t, changes[0].Mask, ChangeTypeRemove)\n\t\trequire.Equal(t, changes[1].Mask, ChangeTypeMove)\n\t\trequire.Equal(t, changes[2].Mask, ChangeTypeModify)\n\t\trequire.Equal(t, changes[3].Mask, ChangeTypeAdd)\n\n\t\tchange := CombineChanges(changes)\n\t\trequire.Equal(t, change.MovedTo, \"\")\n\t\trequire.Equal(t, change.WasPreviouslyAt, \"/x\")\n\t\trequire.Equal(\n\t\t\tt,\n\t\t\tchange.Mask,\n\t\t\tChangeTypeRemove|ChangeTypeMove|ChangeTypeModify|ChangeTypeAdd,\n\t\t)\n\t})\n}\n\nfunc TestChangeCombineMoveBackAndForth(t *testing.T) {\n\tc.WithDummyLinker(t, func(lkr *c.Linker) {\n\t\tx := c.MustTouch(t, lkr, \"/x\", 1)\n\t\tc.MustCommit(t, lkr, \"1\")\n\t\ty := c.MustMove(t, lkr, x, \"/y\")\n\t\tc.MustCommit(t, lkr, \"2\")\n\t\txx := c.MustMove(t, lkr, y, \"/x\")\n\t\tc.MustCommit(t, lkr, \"3\")\n\n\t\tstatus, err := lkr.Status()\n\t\trequire.Nil(t, err)\n\n\t\tchanges, err := History(lkr, xx, status, nil)\n\t\trequire.Nil(t, err)\n\t\trequire.Len(t, changes, 4)\n\t\trequire.Equal(t, changes[0].Mask, ChangeTypeNone)\n\t\trequire.Equal(t, changes[1].Mask, ChangeTypeMove)\n\t\trequire.Equal(t, changes[2].Mask, ChangeTypeMove)\n\t\trequire.Equal(t, changes[3].Mask, ChangeTypeAdd)\n\n\t\tchange := CombineChanges(changes)\n\t\trequire.Equal(t, \"/x\", change.WasPreviouslyAt)\n\t\trequire.Equal(t, \"\", change.MovedTo)\n\t\trequire.Equal(t, ChangeTypeAdd, change.Mask)\n\t})\n}\n\nfunc TestChangeRemoveAndReadd(t *testing.T) {\n\tc.WithDummyLinker(t, func(lkr *c.Linker) {\n\t\tx := c.MustTouch(t, lkr, \"/x\", 1)\n\t\tc.MustCommit(t, lkr, \"1\")\n\t\tc.MustRemove(t, lkr, x)\n\t\tc.MustCommit(t, lkr, \"2\")\n\t\txx := c.MustTouch(t, lkr, \"/x\", 2)\n\t\tc.MustCommit(t, lkr, \"3\")\n\n\t\tstatus, err := lkr.Status()\n\t\trequire.Nil(t, err)\n\n\t\tchanges, err := History(lkr, xx, status, nil)\n\t\trequire.Nil(t, err)\n\t\trequire.Len(t, changes, 4)\n\t\trequire.Equal(t, changes[0].Mask, ChangeTypeNone)\n\t\trequire.Equal(t, changes[1].Mask, ChangeTypeAdd|ChangeTypeModify)\n\t\trequire.Equal(t, changes[2].Mask, ChangeTypeRemove)\n\t\trequire.Equal(t, changes[3].Mask, ChangeTypeAdd)\n\n\t\tchange := CombineChanges(changes)\n\t\trequire.Equal(t, \"\", change.MovedTo)\n\t\trequire.Equal(t, ChangeTypeAdd|ChangeTypeModify, change.Mask)\n\t})\n}\n\nfunc TestChangeReplay(t *testing.T) {\n\ttcs := []struct {\n\t\tname string\n\t\tsetup func(t *testing.T, lkrSrc, lkrDst *c.Linker) n.ModNode\n\t\tcheck func(t *testing.T, lkrSrc, lkrDst *c.Linker, srcNd n.ModNode)\n\t}{\n\t\t{\n\t\t\tname: \"basic-add\",\n\t\t\tsetup: func(t *testing.T, lkrSrc, lkrDst *c.Linker) n.ModNode {\n\t\t\t\treturn c.MustTouch(t, lkrSrc, \"/x\", 1)\n\t\t\t},\n\t\t\tcheck: func(t *testing.T, lkrSrc, lkrDst *c.Linker, srcNd n.ModNode) {\n\t\t\t\tdstX, err := lkrDst.LookupFile(\"/x\")\n\t\t\t\trequire.Nil(t, err)\n\n\t\t\t\trequire.Equal(t, dstX.Size(), srcNd.Size())\n\t\t\t\trequire.Equal(t, dstX.TreeHash(), srcNd.TreeHash())\n\t\t\t\trequire.Equal(t, dstX.BackendHash(), srcNd.BackendHash())\n\t\t\t\trequire.Equal(t, dstX.ContentHash(), srcNd.ContentHash())\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"basic-modify\",\n\t\t\tsetup: func(t *testing.T, lkrSrc, lkrDst *c.Linker) n.ModNode {\n\t\t\t\tc.MustTouch(t, lkrDst, \"/x\", 0)\n\t\t\t\treturn c.MustTouch(t, lkrSrc, \"/x\", 1)\n\t\t\t},\n\t\t\tcheck: func(t *testing.T, lkrSrc, lkrDst *c.Linker, srcNd n.ModNode) {\n\t\t\t\tdstX, err := lkrDst.LookupFile(\"/x\")\n\t\t\t\trequire.Nil(t, err)\n\n\t\t\t\trequire.Equal(t, dstX.Size(), srcNd.Size())\n\t\t\t\trequire.Equal(t, dstX.TreeHash(), srcNd.TreeHash())\n\t\t\t\trequire.Equal(t, dstX.BackendHash(), srcNd.BackendHash())\n\t\t\t\trequire.Equal(t, dstX.ContentHash(), srcNd.ContentHash())\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"basic-remove\",\n\t\t\tsetup: func(t *testing.T, lkrSrc, lkrDst *c.Linker) n.ModNode {\n\t\t\t\tc.MustTouch(t, lkrDst, \"/x\", 1)\n\t\t\t\tsrcX := c.MustTouch(t, lkrSrc, \"/x\", 1)\n\t\t\t\treturn c.MustRemove(t, lkrSrc, srcX)\n\t\t\t},\n\t\t\tcheck: func(t *testing.T, lkrSrc, lkrDst *c.Linker, srcNd n.ModNode) {\n\t\t\t\t// it's enough to assert that it's a ghost now:\n\t\t\t\t_, err := lkrDst.LookupGhost(\"/x\")\n\t\t\t\trequire.Nil(t, err)\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"basic-move\",\n\t\t\tsetup: func(t *testing.T, lkrSrc, lkrDst *c.Linker) n.ModNode {\n\t\t\t\tc.MustTouch(t, lkrDst, \"/x\", 1)\n\t\t\t\tsrcX := c.MustTouch(t, lkrSrc, \"/x\", 1)\n\t\t\t\tc.MustCommit(t, lkrSrc, \"move\")\n\t\t\t\treturn c.MustMove(t, lkrSrc, srcX, \"/y\")\n\t\t\t},\n\t\t\tcheck: func(t *testing.T, lkrSrc, lkrDst *c.Linker, srcNd n.ModNode) {\n\t\t\t\t// it's enough to assert that it's a ghost now:\n\t\t\t\t_, err := lkrDst.LookupGhost(\"/x\")\n\t\t\t\trequire.Nil(t, err)\n\n\t\t\t\t_, err = lkrDst.LookupFile(\"/y\")\n\t\t\t\trequire.Nil(t, err)\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"basic-all\",\n\t\t\tsetup: func(t *testing.T, lkrSrc, lkrDst *c.Linker) n.ModNode {\n\t\t\t\tc.MustTouch(t, lkrDst, \"/x\", 1)\n\n\t\t\t\tsrcX := c.MustTouch(t, lkrSrc, \"/x\", 1)\n\t\t\t\tc.MustCommit(t, lkrSrc, \"touch\")\n\t\t\t\tsrcY := c.MustMove(t, lkrSrc, srcX, \"/y\").(*n.File)\n\t\t\t\tc.MustCommit(t, lkrSrc, \"move\")\n\t\t\t\treturn c.MustRemove(t, lkrSrc, srcY)\n\t\t\t},\n\t\t\tcheck: func(t *testing.T, lkrSrc, lkrDst *c.Linker, srcNd n.ModNode) {\n\t\t\t\t// it's enough to assert that it's a ghost now:\n\t\t\t\t_, err := lkrDst.LookupGhost(\"/x\")\n\t\t\t\trequire.Nil(t, err)\n\n\t\t\t\t_, err = lkrDst.LookupGhost(\"/y\")\n\t\t\t\trequire.Nil(t, err)\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"basic-mkdir\",\n\t\t\tsetup: func(t *testing.T, lkrSrc, lkrDst *c.Linker) n.ModNode {\n\t\t\t\treturn c.MustMkdir(t, lkrSrc, \"/sub\")\n\t\t\t},\n\t\t\tcheck: func(t *testing.T, lkrSrc, lkrDst *c.Linker, srcNd n.ModNode) {\n\t\t\t\tdir, err := lkrDst.LookupDirectory(\"/sub\")\n\t\t\t\trequire.Nil(t, err)\n\t\t\t\trequire.Equal(t, dir.Path(), \"/sub\")\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"edge-conflicting-types\",\n\t\t\tsetup: func(t *testing.T, lkrSrc, lkrDst *c.Linker) n.ModNode {\n\t\t\t\t// Directory and file:\n\t\t\t\tc.MustMkdir(t, lkrDst, \"/sub\")\n\t\t\t\treturn c.MustTouch(t, lkrSrc, \"/sub\", 1)\n\t\t\t},\n\t\t\tcheck: func(t *testing.T, lkrSrc, lkrDst *c.Linker, srcNd n.ModNode) {\n\t\t\t\t// The directory was purged and the file should appear:\n\t\t\t\t// The policy here is \"trust the remote, it's his metadata\"\n\t\t\t\t_, err := lkrDst.LookupFile(\"/sub\")\n\t\t\t\trequire.Nil(t, err)\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"edge-modified-ghost\",\n\t\t\tsetup: func(t *testing.T, lkrSrc, lkrDst *c.Linker) n.ModNode {\n\t\t\t\tsrcX := c.MustTouch(t, lkrSrc, \"/x\", 1)\n\t\t\t\tc.MustCommit(t, lkrSrc, \"1\")\n\t\t\t\tc.MustModify(t, lkrSrc, srcX, 2)\n\t\t\t\tc.MustCommit(t, lkrSrc, \"2\")\n\t\t\t\treturn c.MustRemove(t, lkrSrc, srcX)\n\t\t\t},\n\t\t\tcheck: func(t *testing.T, lkrSrc, lkrDst *c.Linker, srcNd n.ModNode) {\n\t\t\t\t_, err := lkrDst.LookupGhost(\"/x\")\n\t\t\t\trequire.Nil(t, err)\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"edge-mkdir-existing\",\n\t\t\tsetup: func(t *testing.T, lkrSrc, lkrDst *c.Linker) n.ModNode {\n\t\t\t\tc.MustMkdir(t, lkrDst, \"/sub\")\n\t\t\t\treturn c.MustMkdir(t, lkrSrc, \"/sub\")\n\t\t\t},\n\t\t\tcheck: func(t *testing.T, lkrSrc, lkrDst *c.Linker, srcNd n.ModNode) {\n\t\t\t\tdir, err := lkrDst.LookupDirectory(\"/sub\")\n\t\t\t\trequire.Nil(t, err)\n\t\t\t\trequire.Equal(t, dir.Path(), \"/sub\")\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"edge-mkdir-existing-non-empty\",\n\t\t\tsetup: func(t *testing.T, lkrSrc, lkrDst *c.Linker) n.ModNode {\n\t\t\t\tc.MustMkdir(t, lkrDst, \"/sub\")\n\t\t\t\tc.MustTouch(t, lkrDst, \"/sub/x\", 1)\n\t\t\t\treturn c.MustMkdir(t, lkrSrc, \"/sub\")\n\t\t\t},\n\t\t\tcheck: func(t *testing.T, lkrSrc, lkrDst *c.Linker, srcNd n.ModNode) {\n\t\t\t\tdir, err := lkrDst.LookupDirectory(\"/sub\")\n\t\t\t\trequire.Nil(t, err)\n\t\t\t\trequire.Equal(t, dir.Path(), \"/sub\")\n\n\t\t\t\tdstX, err := lkrDst.LookupFile(\"/sub/x\")\n\t\t\t\trequire.Nil(t, err)\n\t\t\t\trequire.Equal(t, dstX.Path(), \"/sub/x\")\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tcs {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tc.WithLinkerPair(t, func(lkrSrc, lkrDst *c.Linker) {\n\t\t\t\tsrcNd := tc.setup(t, lkrSrc, lkrDst)\n\t\t\t\tsrcHead := c.MustCommit(t, lkrSrc, \"post setup\")\n\n\t\t\t\tsrcChanges, err := History(lkrSrc, srcNd, srcHead, nil)\n\t\t\t\trequire.Nil(t, err)\n\n\t\t\t\tch := CombineChanges(srcChanges)\n\t\t\t\trequire.Nil(t, ch.Replay(lkrDst))\n\n\t\t\t\ttc.check(t, lkrSrc, lkrDst, srcNd)\n\t\t\t})\n\t\t})\n\t}\n}\n" }, { "path": "catfs/vcs/debug.go", "content": "package vcs\n\nimport (\n\t\"fmt\"\n)\n\nconst (\n\tprintDebug = false\n)\n\nfunc debug(args ...interface{}) {\n\tif printDebug {\n\t\tfmt.Println(args...)\n\t}\n}\n\nfunc debugf(spec string, args ...interface{}) {\n\tif printDebug {\n\t\tfmt.Printf(spec, args...)\n\t}\n}\n" }, { "path": "catfs/vcs/diff.go", "content": "package vcs\n\nimport (\n\tc \"github.com/sahib/brig/catfs/core\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n)\n\n// DiffPair is a pair of nodes that have a relation in regard of a change. The\n// change is described by the masks.\ntype DiffPair struct {\n\tSrc n.ModNode\n\tDst n.ModNode\n\tSrcMask ChangeType\n\tDstMask ChangeType\n}\n\n// Diff describes a difference between two commits.\ntype Diff struct {\n\tcfg *SyncOptions\n\n\t// Nodes that were added from remote.\n\tAdded []n.ModNode\n\n\t// Nodes that were removed on remote side.\n\tRemoved []n.ModNode\n\n\t// Nodes (of us) that are missing on the remote side.\n\tMissing []n.ModNode\n\n\t// Nodes from remote that were ignored.\n\tIgnored []n.ModNode\n\n\t// Nodes that were only moved (but nothing else)\n\tMoved []DiffPair\n\n\t// Merged contains nodes where sync is able to combine changes\n\t// on both sides (i.e. one side moved, another modified)\n\tMerged []DiffPair\n\n\t// Conflict contains nodes where sync was not able to combine\n\t// the changes made on both sides.\n\tConflict []DiffPair\n}\n\nfunc (df *Diff) handleAdd(src n.ModNode) error {\n\tdf.Added = append(df.Added, src)\n\treturn nil\n}\nfunc (df *Diff) handleRemove(dst n.ModNode) error {\n\tif df.cfg.IgnoreDeletes {\n\t\tdf.Ignored = append(df.Ignored, dst)\n\t\treturn nil\n\t}\n\n\tdf.Removed = append(df.Removed, dst)\n\treturn nil\n}\n\nfunc (df *Diff) handleMissing(dst n.ModNode) error {\n\t// Handle missing files like \"removed\" for diff.\n\tdf.Missing = append(df.Missing, dst)\n\treturn nil\n}\n\nfunc (df *Diff) handleTypeConflict(src, dst n.ModNode) error {\n\tdf.Ignored = append(df.Ignored, dst)\n\treturn nil\n}\n\nfunc (df *Diff) handleConflictNode(nd n.ModNode) error {\n\tdf.Ignored = append(df.Ignored, nd)\n\treturn nil\n}\n\nfunc (df *Diff) handleMove(src, dst n.ModNode) error {\n\tdf.Moved = append(df.Moved, DiffPair{\n\t\tSrc: src,\n\t\tDst: dst,\n\t\tSrcMask: ChangeType(0),\n\t\tDstMask: ChangeType(0),\n\t})\n\n\treturn nil\n}\n\nfunc (df *Diff) handleConflict(src, dst n.ModNode, srcMask, dstMask ChangeType) error {\n\tdf.Conflict = append(df.Conflict, DiffPair{\n\t\tSrc: src,\n\t\tDst: dst,\n\t\tSrcMask: srcMask,\n\t\tDstMask: dstMask,\n\t})\n\n\treturn nil\n}\n\nfunc (df *Diff) handleMerge(src, dst n.ModNode, srcMask, dstMask ChangeType) error {\n\tdf.Merged = append(df.Merged, DiffPair{\n\t\tSrc: src,\n\t\tDst: dst,\n\t\tSrcMask: srcMask,\n\t\tDstMask: dstMask,\n\t})\n\n\treturn nil\n}\n\n// MakeDiff show the differences between two linkers.\n//\n// Internally it works like Sync() but does not modify anything and just\n// merely records what the algorithm decided to do.\nfunc MakeDiff(lkrSrc, lkrDst *c.Linker, headSrc, headDst *n.Commit, cfg *SyncOptions) (*Diff, error) {\n\tif cfg == nil {\n\t\tcfg = defaultSyncConfig\n\t}\n\n\tdiff := &Diff{cfg: cfg}\n\trsv, err := newResolver(lkrSrc, lkrDst, headSrc, headDst, diff)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := rsv.resolve(); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn diff, nil\n}\n" }, { "path": "catfs/vcs/diff_test.go", "content": "package vcs\n\nimport (\n\t\"testing\"\n\n\tc \"github.com/sahib/brig/catfs/core\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc setupDiffBasicSrcFile(t *testing.T, lkrSrc, lkrDst *c.Linker) {\n\tc.MustTouch(t, lkrSrc, \"/x.png\", 1)\n\tc.MustTouch(t, lkrDst, \"/y.png\", 2)\n}\n\nfunc checkDiffBasicSrcFileForward(t *testing.T, lkrSrc, lkrDst *c.Linker, diff *Diff) {\n\trequire.Empty(t, diff.Removed)\n\trequire.Empty(t, diff.Conflict)\n\trequire.Empty(t, diff.Ignored)\n\trequire.Empty(t, diff.Merged)\n\n\trequire.Len(t, diff.Added, 1)\n\trequire.Equal(t, \"/x.png\", diff.Added[0].Path())\n\n\trequire.Len(t, diff.Missing, 1)\n\trequire.Equal(t, \"/y.png\", diff.Missing[0].Path())\n}\n\nfunc checkDiffBasicSrcFileBackward(t *testing.T, lkrSrc, lkrDst *c.Linker, diff *Diff) {\n\trequire.Empty(t, diff.Conflict)\n\trequire.Empty(t, diff.Ignored)\n\trequire.Empty(t, diff.Merged)\n\trequire.Empty(t, diff.Removed)\n\n\trequire.Len(t, diff.Added, 1)\n\trequire.Equal(t, \"/y.png\", diff.Added[0].Path())\n\n\trequire.Len(t, diff.Missing, 1)\n\trequire.Equal(t, \"/x.png\", diff.Missing[0].Path())\n}\n\n///////////////\n\nfunc assertDiffIsEmpty(t *testing.T, diff *Diff) {\n\trequire.Empty(t, diff.Added)\n\trequire.Empty(t, diff.Removed)\n\trequire.Empty(t, diff.Conflict)\n\trequire.Empty(t, diff.Ignored)\n\trequire.Empty(t, diff.Merged)\n\trequire.Empty(t, diff.Missing)\n\n}\n\nfunc TestDiff(t *testing.T) {\n\ttcs := []struct {\n\t\tname string\n\t\tsetup func(t *testing.T, lkrSrc, lkrDst *c.Linker)\n\t\tcheckForward func(t *testing.T, lkrSrc, lkrDst *c.Linker, diff *Diff)\n\t\tcheckBackward func(t *testing.T, lkrSrc, lkrDst *c.Linker, diff *Diff)\n\t}{\n\t\t{\n\t\t\tname: \"basic-file-on-both-sides\",\n\t\t\tsetup: setupDiffBasicSrcFile,\n\t\t\tcheckForward: checkDiffBasicSrcFileForward,\n\t\t\tcheckBackward: checkDiffBasicSrcFileBackward,\n\t\t},\n\t}\n\n\tfor _, tc := range tcs {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tc.WithLinkerPair(t, func(lkrSrc, lkrDst *c.Linker) {\n\t\t\t\tc.MustTouch(t, lkrSrc, \"/README.md\", 42)\n\t\t\t\tc.MustTouch(t, lkrDst, \"/README.md\", 42)\n\n\t\t\t\tc.MustCommitIfPossible(t, lkrDst, \"setup dst\")\n\t\t\t\tc.MustCommitIfPossible(t, lkrSrc, \"setup src\")\n\n\t\t\t\ttc.setup(t, lkrSrc, lkrDst)\n\n\t\t\t\tsrcStatus, err := lkrSrc.Status()\n\t\t\t\trequire.Nil(t, err)\n\n\t\t\t\tdstStatus, err := lkrDst.Status()\n\t\t\t\trequire.Nil(t, err)\n\n\t\t\t\tdiff, err := MakeDiff(lkrSrc, lkrDst, srcStatus, dstStatus, nil)\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Fatalf(\"diff forward failed: %v\", err)\n\t\t\t\t}\n\n\t\t\t\ttc.checkForward(t, lkrSrc, lkrDst, diff)\n\n\t\t\t\tdiff, err = MakeDiff(lkrDst, lkrSrc, dstStatus, srcStatus, nil)\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Fatalf(\"diff backward failed: %v\", err)\n\t\t\t\t}\n\n\t\t\t\ttc.checkBackward(t, lkrSrc, lkrDst, diff)\n\n\t\t\t\t// Checking the same commit should always result into an empty diff:\n\t\t\t\t// We could of course cheat and check the hash to be equal,\n\t\t\t\t// but this is helpful to validate the implementation.\n\t\t\t\tdiff, err = MakeDiff(lkrSrc, lkrSrc, srcStatus, srcStatus, nil)\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Fatalf(\"diff equal src failed: %v\", err)\n\t\t\t\t}\n\n\t\t\t\tassertDiffIsEmpty(t, diff)\n\n\t\t\t\tdiff, err = MakeDiff(lkrDst, lkrDst, dstStatus, dstStatus, nil)\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Fatalf(\"diff equal dst failed: %v\", err)\n\t\t\t\t}\n\n\t\t\t\tassertDiffIsEmpty(t, diff)\n\t\t\t})\n\t\t})\n\t}\n}\n\nfunc TestDiffWithSameLinker(t *testing.T) {\n\tc.WithDummyLinker(t, func(lkr *c.Linker) {\n\t\tc.MustMkdir(t, lkr, \"/old/sub/\")\n\t\tc.MustTouchAndCommit(t, lkr, \"/old/sub/x\", 1)\n\n\t\tc.MustMove(t, lkr, c.MustLookupDirectory(t, lkr, \"/old\"), \"/new\")\n\n\t\t// Fetch current head and status:\n\t\thead, err := lkr.Head()\n\t\trequire.Nil(t, err)\n\n\t\tstatus, err := lkr.Status()\n\t\trequire.Nil(t, err)\n\n\t\t// log.SetOutput(os.Stderr)\n\t\t// log.SetLevel(log.DebugLevel)\n\n\t\tdiff, err := MakeDiff(lkr, lkr, head, status, nil)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"diff forward failed: %v\", err)\n\t\t}\n\n\t\trequire.Empty(t, diff.Added)\n\t\trequire.Empty(t, diff.Removed)\n\t\trequire.Empty(t, diff.Ignored)\n\t\trequire.Empty(t, diff.Conflict)\n\t\trequire.Empty(t, diff.Merged)\n\t\trequire.Empty(t, diff.Missing)\n\n\t\trequire.Len(t, diff.Moved, 1)\n\t\trequire.Equal(t, diff.Moved[0].Src.Path(), \"/old\")\n\t\trequire.Equal(t, diff.Moved[0].Dst.Path(), \"/new\")\n\n\t\tdiff, err = MakeDiff(lkr, lkr, status, head, nil)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"diff backward failed: %v\", err)\n\t\t}\n\n\t\trequire.Empty(t, diff.Added)\n\t\trequire.Empty(t, diff.Removed)\n\t\trequire.Empty(t, diff.Ignored)\n\t\trequire.Empty(t, diff.Conflict)\n\t\trequire.Empty(t, diff.Merged)\n\t\trequire.Empty(t, diff.Missing)\n\n\t\trequire.Len(t, diff.Moved, 1)\n\t\trequire.Equal(t, diff.Moved[0].Dst.Path(), \"/old\")\n\t\trequire.Equal(t, diff.Moved[0].Src.Path(), \"/new\")\n\n\t\tdiff, err = MakeDiff(lkr, lkr, status, status, nil)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"diff equal head: %v\", err)\n\t\t}\n\n\t\tassertDiffIsEmpty(t, diff)\n\n\t\tdiff, err = MakeDiff(lkr, lkr, status, status, nil)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"diff equal status: %v\", err)\n\t\t}\n\n\t\tassertDiffIsEmpty(t, diff)\n\t})\n}\n" }, { "path": "catfs/vcs/history.go", "content": "package vcs\n\nimport (\n\t\"fmt\"\n\t\"path\"\n\t\"strings\"\n\n\te \"github.com/pkg/errors\"\n\tc \"github.com/sahib/brig/catfs/core\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// HistoryWalker provides a way to iterate over all changes a single Node had.\n// It is capable of tracking a file even over multiple moves.\n//\n// The API is loosely modeled after bufio.Scanner and can be used like this:\n//\n// head, _ := lkr.Head()\n// nd, _ := lkr.LookupFile(\"/x\")\n// walker := NewHistoryWalker(lkr, head, nd)\n// for walker.Next() {\n// walker.Change()\n// }\n//\n// if err := walker.Error(); err != nil {\n// // Handle errors.\n// }\ntype HistoryWalker struct {\n\tlkr *c.Linker\n\thead *n.Commit\n\tcurr n.ModNode\n\tnext n.ModNode\n\terr error\n\tstate *Change\n}\n\n// NewHistoryWalker will return a new HistoryWalker that will yield changes of\n// `node` starting from the state in `cmt` until the root commit if desired.\n// Note that it is not checked that `node` is actually part of `cmt`.\nfunc NewHistoryWalker(lkr *c.Linker, cmt *n.Commit, node n.ModNode) *HistoryWalker {\n\treturn &HistoryWalker{\n\t\tlkr: lkr,\n\t\thead: cmt,\n\t\tcurr: node,\n\t}\n}\n\n// maskFromState figures out the change mask based on the current state\nfunc (hw *HistoryWalker) maskFromState(curr, next n.ModNode) ChangeType {\n\tmask := ChangeType(0)\n\n\t// Initial state; no succesor known yet to compare too.\n\tif next == nil {\n\t\treturn mask\n\t}\n\n\tisGhostCurr := curr.Type() == n.NodeTypeGhost\n\tisGhostNext := next.Type() == n.NodeTypeGhost\n\n\tcurrHash, err := n.ContentHash(curr)\n\tif err != nil {\n\t\tlog.Warningf(\"history: misisng content hash for %s (curr)\", curr.Path())\n\t\treturn ChangeTypeNone\n\t}\n\n\tnextHash, err := n.ContentHash(next)\n\tif err != nil {\n\t\tlog.Warningf(\"history: misisng content hash for %s (next)\", next.Path())\n\t\treturn ChangeTypeNone\n\t}\n\n\t// If the hash differs, there's likely a modification going on.\n\tif !currHash.Equal(nextHash) {\n\t\tmask |= ChangeTypeModify\n\t}\n\n\tif next.Path() != curr.Path() {\n\t\tmask |= ChangeTypeMove\n\t} else {\n\t\t// If paths did not move, but the current node is a ghost,\n\t\t// then it means that the node was removed in this commit.\n\t\tif isGhostCurr && !isGhostNext {\n\t\t\tmask |= ChangeTypeRemove\n\t\t}\n\n\t\t// ...otherwise the node was re-added in this commit,\n\t\t// but removed previously.\n\t\tif !isGhostCurr && isGhostNext {\n\t\t\tmask |= ChangeTypeAdd\n\t\t}\n\t}\n\n\treturn mask\n}\n\nfunc parentDirectoryForCommit(lkr *c.Linker, cmt *n.Commit, curr n.Node) (*n.Directory, error) {\n\tnextDirPath := path.Dir(curr.Path())\n\tif nextDirPath == \"/\" {\n\t\treturn nil, nil\n\t}\n\n\troot, err := lkr.DirectoryByHash(cmt.Root())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tnd, err := root.Lookup(lkr, nextDirPath)\n\tif err != nil {\n\t\tif ie.IsNoSuchFileError(err) {\n\t\t\treturn nil, nil\n\t\t}\n\n\t\treturn nil, err\n\t}\n\n\tdir, ok := nd.(*n.Directory)\n\tif !ok {\n\t\treturn nil, ie.ErrBadNode\n\t}\n\n\treturn dir, nil\n}\n\n// Check if a node was moved and if so, return the coressponding other half.\n// If it was not moved, this method will return nil, MoveDirNone, nil.\n//\n// This is also supposed to work with moved directories (keep in mind that\n// moving directories will only create a ghost for the moved directory itself,\n// not the children of it):\n//\n// $ tree .\n// a/\n// b/\n// c # a file.\n// $ mv a f\n//\n// For this case we need to go over the parent directories of c (b and f) to\n// find the ghost dir \"a\". From there we can resolve back to \"c\".\nfunc findMovePartner(lkr *c.Linker, head *n.Commit, curr n.Node) (n.Node, c.MoveDir, error) {\n\tprev, direction, err := lkr.MoveMapping(head, curr)\n\tif err != nil {\n\t\treturn nil, c.MoveDirNone, err\n\t}\n\n\tif prev != nil {\n\t\treturn prev, direction, nil\n\t}\n\n\tchildPath := []string{curr.Name()}\n\n\tfor {\n\t\tparentDir, err := parentDirectoryForCommit(lkr, head, curr)\n\t\tif err != nil {\n\t\t\treturn nil, c.MoveDirNone, e.Wrap(err, \"bad parent dir\")\n\t\t}\n\n\t\tif parentDir == nil {\n\t\t\treturn nil, c.MoveDirNone, nil\n\t\t}\n\n\t\tprevDirNd, direction, err := lkr.MoveMapping(head, parentDir)\n\t\tif err != nil {\n\t\t\treturn nil, c.MoveDirNone, err\n\t\t}\n\n\t\t// Advance for next round:\n\t\tcurr = parentDir\n\n\t\tif prevDirNd == nil {\n\t\t\t// This was not moved; remember step for final lookup:\n\t\t\tchildPath = append([]string{parentDir.Name()}, childPath...)\n\t\t\tcontinue\n\t\t}\n\n\t\t// At this point we know that the dir `parentDir` was moved.\n\t\t// Now we have to find the old version of the node and exit this for loop.\n\n\t\tvar prevDir *n.Directory\n\n\t\tswitch prevDirNd.Type() {\n\t\tcase n.NodeTypeDirectory:\n\t\t\t// This case will probably not happen not very often.\n\t\t\t// Most of the time the old node in a mapping is a ghost.\n\t\t\tvar ok bool\n\t\t\tprevDir, ok = prevDirNd.(*n.Directory)\n\t\t\tif !ok {\n\t\t\t\treturn nil, c.MoveDirNone, ie.ErrBadNode\n\t\t\t}\n\t\tcase n.NodeTypeGhost:\n\t\t\t// If it's a ghost we need to unpack it.\n\t\t\tprevDirGhost, ok := prevDirNd.(*n.Ghost)\n\t\t\tif !ok {\n\t\t\t\treturn nil, c.MoveDirNone, e.Wrap(\n\t\t\t\t\tie.ErrBadNode,\n\t\t\t\t\t\"bad previous dir\",\n\t\t\t\t)\n\t\t\t}\n\n\t\t\tprevDir, err = prevDirGhost.OldDirectory()\n\t\t\tif err != nil {\n\t\t\t\treturn nil, c.MoveDirNone, e.Wrap(err, \"bad old directory\")\n\t\t\t}\n\t\tdefault:\n\t\t\treturn nil, c.MoveDirNone, fmt.Errorf(\"unexpected file node\")\n\t\t}\n\n\t\t// By the current logic, the path is still reachable in\n\t\t// the directory the same way before.\n\t\tchild, err := prevDir.Lookup(lkr, strings.Join(childPath, \"/\"))\n\t\tif err != nil {\n\t\t\treturn nil, c.MoveDirNone, err\n\t\t}\n\n\t\treturn child, direction, nil\n\t}\n}\n\nfunc getRealType(nd n.Node) n.NodeType {\n\tif nd.Type() == n.NodeTypeGhost {\n\t\treturn nd.(*n.Ghost).OldNode().Type()\n\t}\n\n\treturn nd.Type()\n}\n\nfunc (hw *HistoryWalker) findReferToPath(prevHeadCommit *n.Commit, prev n.Node) (string, n.Node, error) {\n\tif prev == nil {\n\t\treturn \"\", nil, nil\n\t}\n\n\treferToPath := prev.Path()\n\n\t// Unpack the old ghost before doing anything with it:\n\tif prev.Type() != n.NodeTypeGhost {\n\t\treturn referToPath, prev, nil\n\t}\n\n\tprevGhost, ok := prev.(*n.Ghost)\n\tif !ok {\n\t\treturn \"\", nil, ie.ErrBadNode\n\t}\n\n\tprev = prevGhost.OldNode()\n\n\t// Special case:\n\t// A file has a move partner in this commit,\n\t// but there is no previous node in the commit before.\n\t// We should count this node as added therefore, not moved.\n\tprevRoot, err := hw.lkr.DirectoryByHash(prevHeadCommit.Root())\n\tif err != nil {\n\t\treturn \"\", nil, e.Wrap(err, \"cannot find previous root directory\")\n\t}\n\n\t_, err = prevRoot.Lookup(hw.lkr, prev.Path())\n\tif ie.IsNoSuchFileError(err) {\n\t\tprev = nil\n\t\treferToPath = \"\"\n\t}\n\n\treturn referToPath, prev, nil\n}\n\nfunc (hw *HistoryWalker) findDirectPrev(prevHeadCommit *n.Commit) (n.Node, bool) {\n\tprevRoot, err := hw.lkr.DirectoryByHash(prevHeadCommit.Root())\n\tif err != nil {\n\t\thw.err = e.Wrap(err, \"cannot find previous root directory\")\n\t\treturn nil, false\n\t}\n\n\tprev, err := prevRoot.Lookup(hw.lkr, hw.curr.Path())\n\tif ie.IsNoSuchFileError(err) {\n\t\t// The file did not exist in the previous commit (no ghost!)\n\t\t// It must have been added in this commit.\n\t\thw.state = &Change{\n\t\t\tHead: hw.head,\n\t\t\tMask: ChangeTypeAdd,\n\t\t\tCurr: hw.curr,\n\t\t\tNext: prevHeadCommit,\n\t\t}\n\n\t\t// If curr is a ghost we have a rare case:\n\t\t// The node was added and removed in the same commit.\n\t\tif hw.curr.Type() == n.NodeTypeGhost {\n\t\t\thw.state.Mask |= ChangeTypeRemove\n\t\t}\n\n\t\thw.head = nil\n\t\treturn nil, true\n\t}\n\n\tif err != nil {\n\t\thw.err = e.Wrap(err, \"history: prev root lookup failed\")\n\t\treturn nil, false\n\t}\n\n\treturn prev, false\n}\n\n// Next advances the walker to the next commit.\n// Call State() to get the current state after.\n// If there are no commits left or an error happened,\n// false is returned. True otherwise. You should check\n// after a failing Next() if an error happened via Err()\nfunc (hw *HistoryWalker) Next() bool {\n\tif hw.err != nil {\n\t\treturn false\n\t}\n\n\tif hw.head == nil {\n\t\treturn false\n\t}\n\n\t// Check if this node participated in a move:\n\tprev, direction, err := findMovePartner(hw.lkr, hw.head, hw.curr)\n\tif err != nil {\n\t\thw.err = err\n\t\treturn false\n\t}\n\n\t// Advance to the previous commit:\n\tprevHead, err := hw.head.Parent(hw.lkr)\n\tif err != nil {\n\t\thw.err = err\n\t\treturn false\n\t}\n\n\t// We ran out of commits to check.\n\tif prevHead == nil {\n\t\thw.state = &Change{\n\t\t\tHead: hw.head,\n\t\t\tMask: ChangeTypeAdd,\n\t\t\tCurr: hw.curr,\n\t\t\tNext: nil,\n\t\t}\n\n\t\tif hw.curr.Type() == n.NodeTypeGhost {\n\t\t\thw.state.Mask |= ChangeTypeRemove\n\t\t}\n\n\t\thw.head = nil\n\t\treturn true\n\t}\n\n\tprevHeadCommit, ok := prevHead.(*n.Commit)\n\tif !ok {\n\t\thw.err = e.Wrap(ie.ErrBadNode, \"history: bad commit\")\n\t\treturn false\n\t}\n\n\t// Try to find the node \"prev\" is actually referring to in the old commit:\n\treferToPath, prev, err := hw.findReferToPath(prevHeadCommit, prev)\n\tif err != nil {\n\t\thw.err = e.Wrap(err, \"history: findReferToPath\")\n\t\treturn false\n\t}\n\n\t// Assumption here: The move mapping should only store one move per commit.\n\t// i.e: for move(a, b); a and b should always be in different commits.\n\t// This is enforced by the logic in MakeCommit()\n\tif prev == nil || direction != c.MoveDirSrcToDst {\n\t\t// No valid move mapping found, node was probably not moved.\n\t\t// Assume that we can reach it directly via it's path.\n\t\tvar hasNext bool\n\t\tprev, hasNext = hw.findDirectPrev(prevHeadCommit)\n\t\tif prev == nil {\n\t\t\treturn hasNext\n\t\t}\n\t}\n\n\tprevModNode, ok := prev.(n.ModNode)\n\tif !ok {\n\t\thw.err = e.Wrap(ie.ErrBadNode, \"history: bad mod node\")\n\t\treturn false\n\t}\n\n\t// Pack up the current state:\n\thw.state = &Change{\n\t\tHead: hw.head,\n\t\tMask: hw.maskFromState(hw.curr, prevModNode),\n\t\tCurr: hw.curr,\n\t\tNext: prevHeadCommit,\n\t}\n\n\tif getRealType(prev) != getRealType(hw.curr) {\n\t\t// Edge case: The node changed its types. This can happen when we\n\t\t// remove a file and create a directory in its place.\n\t\thw.state.Mask = ChangeTypeAdd\n\t\thw.state.Next = nil\n\t\treturn false\n\t}\n\n\t// Special case #1: A ghost that still has a move partner.\n\t// This means the node here was moved to `prev` in this commit.\n\tif hw.curr.Type() == n.NodeTypeGhost && direction == c.MoveDirDstToSrc {\n\t\t// Indicate that this node was indeed removed,\n\t\t// but still lives somewhere else.\n\t\thw.state.Mask |= ChangeTypeMove\n\t\thw.state.MovedTo = referToPath\n\t}\n\n\t// Special case #2: A non-ghost that was moved from somewhere.\n\tif hw.curr.Type() != n.NodeTypeGhost && direction == c.MoveDirSrcToDst {\n\t\thw.state.Mask |= ChangeTypeMove\n\t\thw.state.WasPreviouslyAt = referToPath\n\t}\n\n\t// Swap for the next call to Next():\n\thw.curr, hw.next = prevModNode, hw.curr\n\thw.head = prevHeadCommit\n\treturn true\n}\n\n// State returns the current change state.\n// Note that the change may have ChangeTypeNone as Mask if nothing changed.\n// If you only want states where it actually changed, just filter those.\nfunc (hw *HistoryWalker) State() *Change {\n\treturn hw.state\n}\n\n// Err returns the last happened error or nil if none.\nfunc (hw *HistoryWalker) Err() error {\n\treturn hw.err\n}\n\n// History returns a list of `nd`'s states starting with the commit in `start`\n// and stopping at `stop`. `stop` can be nil; in this case all commits will be\n// iterated. The returned list has the most recent change upfront, and the\n// latest change as last element.\nfunc History(lkr *c.Linker, nd n.ModNode, start, stop *n.Commit) ([]*Change, error) {\n\tstates := make([]*Change, 0)\n\twalker := NewHistoryWalker(lkr, start, nd)\n\n\tfor walker.Next() {\n\t\tstate := walker.State()\n\t\tstates = append(states, state)\n\n\t\t// Stop searching when he iterated deep enough:\n\t\tif stop != nil && state.Head.TreeHash().Equal(stop.TreeHash()) {\n\t\t\tbreak\n\t\t}\n\t}\n\n\tif err := walker.Err(); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn states, nil\n}\n" }, { "path": "catfs/vcs/history_test.go", "content": "package vcs\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\tc \"github.com/sahib/brig/catfs/core\"\n\t\"github.com/sahib/brig/catfs/db\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\tlog \"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc init() {\n\tlog.SetLevel(log.WarnLevel)\n}\n\ntype historySetup struct {\n\tcommits []*n.Commit\n\tpaths []string\n\tchanges []ChangeType\n\thead *n.Commit\n\tnode n.ModNode\n}\n\n/////////////// ACTUAL TESTCASES ///////////////\n\nfunc setupHistoryBasic(t *testing.T, lkr *c.Linker) *historySetup {\n\t_, c1 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 1)\n\t_, c2 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 2)\n\tfile, c3 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 3)\n\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to retrieve status: %v\", err)\n\t}\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{status, c3, c2, c1},\n\t\tpaths: []string{\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeNone,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: status,\n\t\tnode: file,\n\t}\n}\n\nfunc setupHistoryBasicHole(t *testing.T, lkr *c.Linker) *historySetup {\n\t_, c1 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 1)\n\t_, c2 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 2)\n\n\t// Needed to have a commit that has changes:\n\tc.MustTouch(t, lkr, \"/other\", 23)\n\tfile, c3 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 2)\n\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to retrieve status: %v\", err)\n\t}\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{status, c3, c2, c1},\n\t\tpaths: []string{\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeNone,\n\t\t\tChangeTypeNone,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: status,\n\t\tnode: file,\n\t}\n}\n\nfunc setupHistoryRemoveImmediately(t *testing.T, lkr *c.Linker) *historySetup {\n\tx := c.MustTouch(t, lkr, \"/x\", 1)\n\tc.MustRemove(t, lkr, x)\n\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to retrieve status: %v\", err)\n\t}\n\n\tghostX, err := lkr.LookupGhost(\"/x\")\n\trequire.Nil(t, err)\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{status},\n\t\tpaths: []string{\n\t\t\t\"/x\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeRemove | ChangeTypeAdd,\n\t\t},\n\t\thead: status,\n\t\tnode: ghostX,\n\t}\n}\n\nfunc setupHistoryRemoved(t *testing.T, lkr *c.Linker) *historySetup {\n\t_, c1 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 1)\n\tfile, c2 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 2)\n\tc.MustRemove(t, lkr, file)\n\tc3 := c.MustCommit(t, lkr, \"after remove\")\n\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to acquire status: %v\", err)\n\t}\n\n\t// removing will copy file and make that a ghost.\n\t// i.e. we need to re-lookup it:\n\tghost, err := lkr.LookupGhost(file.Path())\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to lookup ghost at %s: %v\", file.Path(), err)\n\t}\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{status, c3, c2, c1},\n\t\tpaths: []string{\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeNone,\n\t\t\tChangeTypeRemove,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: status,\n\t\tnode: ghost,\n\t}\n}\n\nfunc setupHistoryMoved(t *testing.T, lkr *c.Linker) *historySetup {\n\t_, c1 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 1)\n\tfile, c2 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 2)\n\tc.MustMove(t, lkr, file, \"/y.png\")\n\tc3 := c.MustCommit(t, lkr, \"post-move\")\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{c3, c2, c1},\n\t\tpaths: []string{\n\t\t\t\"/y.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeMove,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: c3,\n\t\tnode: file,\n\t}\n}\n\nfunc setupHistoryMoveStaging(t *testing.T, lkr *c.Linker) *historySetup {\n\tc.MustTouch(t, lkr, \"/x.png\", 1)\n\t_, c1 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 1)\n\tfile, c2 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 2)\n\tc.MustMove(t, lkr, file, \"/y.png\")\n\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to retrieve status: %v\", err)\n\t}\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{status, c2, c1},\n\t\tpaths: []string{\n\t\t\t\"/y.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeMove,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: status,\n\t\tnode: file,\n\t}\n}\n\nfunc setupMoveInitial(t *testing.T, lkr *c.Linker) *historySetup {\n\tfile := c.MustTouch(t, lkr, \"/x.png\", 1)\n\tc.MustMove(t, lkr, file, \"/y.png\")\n\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to retrieve status: %v\", err)\n\t}\n\n\t// Should act like the node was added as \"y.png\",\n\t// even though it was moved.\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{status},\n\t\tpaths: []string{\n\t\t\t\"/y.png\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: status,\n\t\tnode: file,\n\t}\n}\n\nfunc setupHistoryMoveAndModify(t *testing.T, lkr *c.Linker) *historySetup {\n\t_, c1 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 1)\n\tfile, c2 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 2)\n\n\tnewFile := c.MustMove(t, lkr, file, \"/y.png\")\n\tc.MustModify(t, lkr, newFile.(*n.File), 42)\n\tc3 := c.MustCommit(t, lkr, \"post-move-modify\")\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{c3, c2, c1},\n\t\tpaths: []string{\n\t\t\t\"/y.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeModify | ChangeTypeMove,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: c3,\n\t\tnode: file,\n\t}\n}\n\nfunc setupHistoryMoveAndModifyStage(t *testing.T, lkr *c.Linker) *historySetup {\n\t_, c1 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 1)\n\tfile, c2 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 2)\n\tnewFile := c.MustMove(t, lkr, file, \"/y.png\")\n\tc.MustModify(t, lkr, newFile.(*n.File), 42)\n\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to retrieve status: %v\", err)\n\t}\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{status, c2, c1},\n\t\tpaths: []string{\n\t\t\t\"/y.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeModify | ChangeTypeMove,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: status,\n\t\tnode: file,\n\t}\n}\n\nfunc setupHistoryRemoveReadd(t *testing.T, lkr *c.Linker) *historySetup {\n\t_, c1 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 1)\n\tfile, c2 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 2)\n\tc.MustRemove(t, lkr, file)\n\tc3 := c.MustCommit(t, lkr, \"after remove\")\n\tfile, c4 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 2)\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{c4, c3, c2, c1},\n\t\tpaths: []string{\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeAdd,\n\t\t\tChangeTypeRemove,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: c4,\n\t\tnode: file,\n\t}\n}\n\nfunc setupHistoryRemoveReaddModify(t *testing.T, lkr *c.Linker) *historySetup {\n\t_, c1 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 1)\n\tfile, c2 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 2)\n\tc.MustRemove(t, lkr, file)\n\tc3 := c.MustCommit(t, lkr, \"after remove\")\n\tfile, c4 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 255)\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{c4, c3, c2, c1},\n\t\tpaths: []string{\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeAdd | ChangeTypeModify,\n\t\t\tChangeTypeRemove,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: c4,\n\t\tnode: file,\n\t}\n}\n\nfunc setupHistoryRemoveReaddNoModify(t *testing.T, lkr *c.Linker) *historySetup {\n\t_, c1 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 1)\n\tfile, c2 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 2)\n\tc.MustRemove(t, lkr, file)\n\tc3 := c.MustCommit(t, lkr, \"after remove\")\n\tfile, c4 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 2)\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{c4, c3, c2, c1},\n\t\tpaths: []string{\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeAdd,\n\t\t\tChangeTypeRemove,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: c4,\n\t\tnode: file,\n\t}\n}\n\nfunc setupHistoryMoveCircle(t *testing.T, lkr *c.Linker) *historySetup {\n\t_, c1 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 1)\n\tfile, c2 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 2)\n\tnewFile := c.MustMove(t, lkr, file, \"/y.png\")\n\tc3 := c.MustCommit(t, lkr, \"move to y.png\")\n\tnewOldFile := c.MustMove(t, lkr, newFile, \"/x.png\")\n\tc4 := c.MustCommit(t, lkr, \"move back to x.png\")\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{c4, c3, c2, c1},\n\t\tpaths: []string{\n\t\t\t\"/x.png\",\n\t\t\t\"/y.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeMove,\n\t\t\tChangeTypeMove,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: c4,\n\t\tnode: newOldFile,\n\t}\n}\n\nfunc setupHistoryMoveSamePlaceLeft(t *testing.T, lkr *c.Linker) *historySetup {\n\tx := c.MustTouch(t, lkr, \"/x\", 1)\n\ty := c.MustTouch(t, lkr, \"/y\", 1)\n\tc1 := c.MustCommit(t, lkr, \"pre-move\")\n\n\tc.MustMove(t, lkr, x, \"/z\")\n\tc.MustMove(t, lkr, y, \"/z\")\n\tc2 := c.MustCommit(t, lkr, \"post-move\")\n\n\txGhost, err := lkr.LookupGhost(\"/x\")\n\trequire.Nil(t, err)\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{c2, c1},\n\t\tpaths: []string{\n\t\t\t\"/x\",\n\t\t\t\"/x\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\t// This file was removed, since the destination \"z\"\n\t\t\t// was overwritten by \"y\" and thus we may not count it\n\t\t\t// as moved.\n\t\t\tChangeTypeRemove,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: c2,\n\t\tnode: xGhost,\n\t}\n}\n\nfunc setupHistoryTypeChange(t *testing.T, lkr *c.Linker) *historySetup {\n\tx := c.MustTouch(t, lkr, \"/x\", 1)\n\tc.MustCommit(t, lkr, \"added\")\n\tc.MustRemove(t, lkr, x)\n\tc.MustCommit(t, lkr, \"removed\")\n\tdir := c.MustMkdir(t, lkr, \"/x\")\n\tc3 := c.MustCommit(t, lkr, \"mkdir\")\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{c3},\n\t\tpaths: []string{\n\t\t\t\"/x\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\t// This file was removed, since the destination \"z\"\n\t\t\t// was overwritten by \"y\" and thus we may not count it\n\t\t\t// as moved.\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: c3,\n\t\tnode: dir,\n\t}\n}\n\nfunc setupHistoryMoveSamePlaceRight(t *testing.T, lkr *c.Linker) *historySetup {\n\tx := c.MustTouch(t, lkr, \"/x\", 1)\n\ty := c.MustTouch(t, lkr, \"/y\", 1)\n\tc1 := c.MustCommit(t, lkr, \"pre-move\")\n\n\tc.MustMove(t, lkr, x, \"/z\")\n\tc.MustMove(t, lkr, y, \"/z\")\n\tc2 := c.MustCommit(t, lkr, \"post-move\")\n\n\tyGhost, err := lkr.LookupGhost(\"/y\")\n\trequire.Nil(t, err)\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{c2, c1},\n\t\tpaths: []string{\n\t\t\t\"/y\",\n\t\t\t\"/y\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeMove | ChangeTypeRemove,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: c2,\n\t\tnode: yGhost,\n\t}\n}\n\nfunc setupHistoryMoveAndReaddFromMoved(t *testing.T, lkr *c.Linker) *historySetup {\n\t_, c1 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 1)\n\tfile, c2 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 2)\n\n\tnewFile := c.MustMove(t, lkr, file, \"/y.png\")\n\t_, c3 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 23)\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{c3, c2, c1},\n\t\tpaths: []string{\n\t\t\t\"/y.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeMove,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: c3,\n\t\tnode: newFile,\n\t}\n}\n\nfunc setupHistoryMultipleMovesPerCommit(t *testing.T, lkr *c.Linker) *historySetup {\n\t// Check if we can track multiple moves per commit:\n\tfileX, c1 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 1)\n\tfileY := c.MustMove(t, lkr, fileX, \"/y.png\")\n\tc.MustMove(t, lkr, fileY, \"/z.png\")\n\n\tfileZNew, err := c.Stage(\n\t\tlkr,\n\t\t\"/z.png\",\n\t\th.TestDummy(t, 2),\n\t\th.TestDummy(t, 2),\n\t\tuint64(2),\n\t\t-1,\n\t\tnil,\n\t\ttime.Now(),\n\t\tfalse,\n\t)\n\trequire.Nil(t, err)\n\n\tc2 := c.MustCommit(t, lkr, \"Moved around\")\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{c2, c1},\n\t\tpaths: []string{\n\t\t\t\"/z.png\",\n\t\t\t\"/x.png\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeMove | ChangeTypeModify,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: c2,\n\t\tnode: fileZNew,\n\t}\n}\n\nfunc setupHistoryMultipleMovesInStage(t *testing.T, lkr *c.Linker) *historySetup {\n\t// Check if we can track multiple moves per commit:\n\tfileX, c1 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 1)\n\tfileY := c.MustMove(t, lkr, fileX, \"/y.png\")\n\tfileZ := c.MustMove(t, lkr, fileY, \"/z.png\")\n\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to acquire status: %v\", err)\n\t}\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{status, c1},\n\t\tpaths: []string{\n\t\t\t\"/z.png\",\n\t\t\t\"/x.png\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeMove,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: status,\n\t\tnode: fileZ,\n\t}\n}\n\nfunc setupHistoryMoveAndReaddFromAdded(t *testing.T, lkr *c.Linker) *historySetup {\n\t_, c1 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 1)\n\tfile, c2 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 2)\n\n\tc.MustMove(t, lkr, file, \"/y.png\")\n\tc3 := c.MustCommit(t, lkr, \"move to y.png\")\n\treaddedFile, c4 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 23)\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{c4, c3, c2, c1},\n\t\tpaths: []string{\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t},\n\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeAdd | ChangeTypeModify,\n\t\t\tChangeTypeMove | ChangeTypeRemove,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: c4,\n\t\tnode: readdedFile,\n\t}\n}\n\nfunc setupMoveDirectoryWithChild(t *testing.T, lkr *c.Linker) *historySetup {\n\tdir := c.MustMkdir(t, lkr, \"/sub\")\n\t_, c1 := c.MustTouchAndCommit(t, lkr, \"/sub/x.png\", 1)\n\tfile, c2 := c.MustTouchAndCommit(t, lkr, \"/sub/x.png\", 2)\n\n\tc.MustMove(t, lkr, dir, \"/moved-sub\")\n\tc3 := c.MustCommit(t, lkr, \"moved\")\n\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to get status: %v\", err)\n\t}\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{status, c3, c2, c1},\n\t\tpaths: []string{\n\t\t\t\"/moved-sub/x.png\",\n\t\t\t\"/moved-sub/x.png\",\n\t\t\t\"/sub/x.png\",\n\t\t\t\"/sub/x.png\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeNone,\n\t\t\tChangeTypeMove,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: status,\n\t\tnode: file,\n\t}\n}\n\nfunc setupDirectoryHistory(t *testing.T, lkr *c.Linker) *historySetup {\n\tdir := c.MustMkdir(t, lkr, \"/src\")\n\t_, c1 := c.MustTouchAndCommit(t, lkr, \"/src/x.png\", 1)\n\t_, c2 := c.MustTouchAndCommit(t, lkr, \"/src/x.png\", 2)\n\n\tnewDir := c.MustMove(t, lkr, dir, \"/dst\")\n\tc3 := c.MustCommit(t, lkr, \"move\")\n\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to get status: %v\", err)\n\t}\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{status, c3, c2, c1},\n\t\tpaths: []string{\n\t\t\t\"/dst\",\n\t\t\t\"/dst\",\n\t\t\t\"/src\",\n\t\t\t\"/src\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeNone,\n\t\t\tChangeTypeMove,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: status,\n\t\tnode: newDir,\n\t}\n}\n\nfunc setupGhostHistory(t *testing.T, lkr *c.Linker) *historySetup {\n\t_, c1 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 1)\n\tfile, c2 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 2)\n\tc.MustMove(t, lkr, file, \"/y.png\")\n\tc3 := c.MustCommit(t, lkr, \"move\")\n\n\tghost, err := lkr.LookupGhost(\"/x.png\")\n\trequire.Nil(t, err)\n\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to get status: %v\", err)\n\t}\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{status, c3, c2, c1},\n\t\tpaths: []string{\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t\t\"/x.png\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeNone,\n\t\t\t// The \"ChangeTypeMove\" here is a hint that\n\t\t\t// this ghost was part of a move.\n\t\t\tChangeTypeMove | ChangeTypeRemove,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: status,\n\t\tnode: ghost,\n\t}\n}\n\nfunc setupEdgeRoot(t *testing.T, lkr *c.Linker) *historySetup {\n\tinit, err := lkr.Head()\n\tif err != nil {\n\t\tt.Fatalf(\"could not get head: %v\", err)\n\t}\n\n\t_, c1 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 1)\n\t_, c2 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 2)\n\t_, c3 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 3)\n\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\tt.Fatalf(\"failed to get status: %v\", err)\n\t}\n\n\troot, err := lkr.Root()\n\tif err != nil {\n\t\tt.Fatalf(\"failed to retrieve root: %v\", err)\n\t}\n\n\treturn &historySetup{\n\t\tcommits: []*n.Commit{status, c3, c2, c1, init},\n\t\tpaths: []string{\n\t\t\t\"/\",\n\t\t\t\"/\",\n\t\t\t\"/\",\n\t\t\t\"/\",\n\t\t\t\"/\",\n\t\t},\n\t\tchanges: []ChangeType{\n\t\t\tChangeTypeNone,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeModify,\n\t\t\tChangeTypeAdd,\n\t\t},\n\t\thead: status,\n\t\tnode: root,\n\t}\n}\n\ntype setupFunc func(t *testing.T, lkr *c.Linker) *historySetup\n\n// Registry bank for all testcases:\nfunc TestHistoryWalker(t *testing.T) {\n\ttcs := []struct {\n\t\tname string\n\t\tsetup setupFunc\n\t}{\n\t\t{\n\t\t\tname: \"no-frills\",\n\t\t\tsetup: setupHistoryBasic,\n\t\t}, {\n\t\t\tname: \"holes\",\n\t\t\tsetup: setupHistoryBasicHole,\n\t\t}, {\n\t\t\tname: \"remove-it\",\n\t\t\tsetup: setupHistoryRemoved,\n\t\t}, {\n\t\t\tname: \"remove-readd-simple\",\n\t\t\tsetup: setupHistoryRemoveReadd,\n\t\t}, {\n\t\t\tname: \"remove-immedidately\",\n\t\t\tsetup: setupHistoryRemoveImmediately,\n\t\t}, {\n\t\t\tname: \"remove-readd-modify\",\n\t\t\tsetup: setupHistoryRemoveReaddModify,\n\t\t}, {\n\t\t\tname: \"remove-readd-no-modify\",\n\t\t\tsetup: setupHistoryRemoveReaddNoModify,\n\t\t}, {\n\t\t\tname: \"move-once\",\n\t\t\tsetup: setupHistoryMoved,\n\t\t}, {\n\t\t\tname: \"move-multiple-per-commit\",\n\t\t\tsetup: setupHistoryMultipleMovesPerCommit,\n\t\t}, {\n\t\t\tname: \"move-multiple-per-stage\",\n\t\t\tsetup: setupHistoryMultipleMovesInStage,\n\t\t}, {\n\t\t\tname: \"move-once-stage\",\n\t\t\tsetup: setupHistoryMoveStaging,\n\t\t}, {\n\t\t\tname: \"move-initial\",\n\t\t\tsetup: setupMoveInitial,\n\t\t}, {\n\t\t\tname: \"move-modify\",\n\t\t\tsetup: setupHistoryMoveAndModify,\n\t\t}, {\n\t\t\tname: \"move-to-same-place-left\",\n\t\t\tsetup: setupHistoryMoveSamePlaceLeft,\n\t\t}, {\n\t\t\tname: \"move-to-same-place-right\",\n\t\t\tsetup: setupHistoryMoveSamePlaceRight,\n\t\t}, {\n\t\t\tname: \"move-modify-stage\",\n\t\t\tsetup: setupHistoryMoveAndModifyStage,\n\t\t}, {\n\t\t\tname: \"move-circle\",\n\t\t\tsetup: setupHistoryMoveCircle,\n\t\t}, {\n\t\t\tname: \"move-readd-from-moved-perspective\",\n\t\t\tsetup: setupHistoryMoveAndReaddFromMoved,\n\t\t}, {\n\t\t\tname: \"move-readd-from-readded-perspective\",\n\t\t\tsetup: setupHistoryMoveAndReaddFromAdded,\n\t\t}, {\n\t\t\tname: \"move-directory-with-child\",\n\t\t\tsetup: setupMoveDirectoryWithChild,\n\t\t}, {\n\t\t\tname: \"directory-simple\",\n\t\t\tsetup: setupDirectoryHistory,\n\t\t}, {\n\t\t\tname: \"ghost-simple\",\n\t\t\tsetup: setupGhostHistory,\n\t\t}, {\n\t\t\tname: \"edge-root\",\n\t\t\tsetup: setupEdgeRoot,\n\t\t}, {\n\t\t\tname: \"edge-type-change\",\n\t\t\tsetup: setupHistoryTypeChange,\n\t\t},\n\t}\n\n\tfor _, tc := range tcs {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tc.WithDummyLinker(t, func(lkr *c.Linker) {\n\t\t\t\tsetup := tc.setup(t, lkr)\n\t\t\t\ttestHistoryRunner(t, lkr, setup)\n\t\t\t})\n\t\t})\n\t}\n}\n\n// Actual test runner:\nfunc testHistoryRunner(t *testing.T, lkr *c.Linker, setup *historySetup) {\n\tidx := 0\n\twalker := NewHistoryWalker(lkr, setup.head, setup.node)\n\tfor walker.Next() {\n\t\tstate := walker.State()\n\t\t// fmt.Println(\"TYPE\", state.Mask)\n\t\t// fmt.Println(\"HEAD\", state.Head)\n\t\t// fmt.Println(\"NEXT\", state.Next)\n\t\t// fmt.Println(\"===\")\n\n\t\tif idx >= len(setup.paths) {\n\t\t\tt.Fatalf(\"more history entries than expected\")\n\t\t}\n\n\t\tif setup.paths[idx] != state.Curr.Path() {\n\t\t\tt.Fatalf(\n\t\t\t\t\"Wrong path at index `%d`: %s (want: %s)\",\n\t\t\t\tidx+1, state.Curr.Path(), setup.paths[idx],\n\t\t\t)\n\t\t}\n\n\t\tif state.Mask != setup.changes[idx] {\n\t\t\tt.Errorf(\n\t\t\t\t\"%d: Wrong type of state: %v (want: %s)\",\n\t\t\t\tidx,\n\t\t\t\tstate.Mask,\n\t\t\t\tsetup.changes[idx],\n\t\t\t)\n\t\t}\n\n\t\tif !setup.commits[idx].TreeHash().Equal(state.Head.TreeHash()) {\n\t\t\tt.Fatalf(\"Hash in commit differs\")\n\t\t}\n\n\t\tidx++\n\t}\n\n\tif err := walker.Err(); err != nil {\n\t\tt.Fatalf(\n\t\t\t\"walker failed at index (%d/%d): %v\",\n\t\t\tidx+1,\n\t\t\tlen(setup.commits),\n\t\t\terr,\n\t\t)\n\t}\n}\n\n// Test the History() utility based on HistoryWalker.\nfunc TestHistoryUtil(t *testing.T) {\n\tc.WithDummyLinker(t, func(lkr *c.Linker) {\n\t\tc1File, c1 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 1)\n\t\tc1File = c1File.Copy(c1File.Inode()).(*n.File)\n\n\t\tc2File, c2 := c.MustTouchAndCommit(t, lkr, \"/x.png\", 2)\n\t\tc2File = c2File.Copy(c2File.Inode()).(*n.File)\n\n\t\tc3File := c.MustMove(t, lkr, c2File.Copy(c2File.Inode()), \"/y.png\")\n\t\tc3File = c3File.Copy(c3File.Inode()).(*n.File)\n\t\tc3 := c.MustCommit(t, lkr, \"move to y.png\")\n\n\t\tc4File, c4 := c.MustTouchAndCommit(t, lkr, \"/y.png\", 23)\n\t\tc4File = c4File.Copy(c4File.Inode()).(*n.File)\n\n\t\tstates, err := History(lkr, c4File, c4, nil)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"History without stop commit failed: %v\", err)\n\t\t}\n\n\t\texpected := []*Change{\n\t\t\t{\n\t\t\t\tHead: c4,\n\t\t\t\tCurr: c4File,\n\t\t\t\tMask: ChangeTypeModify,\n\t\t\t}, {\n\t\t\t\tHead: c3,\n\t\t\t\tCurr: c3File,\n\t\t\t\tMask: ChangeTypeMove,\n\t\t\t}, {\n\t\t\t\tHead: c2,\n\t\t\t\tCurr: c2File,\n\t\t\t\tMask: ChangeTypeModify,\n\t\t\t}, {\n\t\t\t\tHead: c1,\n\t\t\t\tCurr: c1File,\n\t\t\t\tMask: ChangeTypeAdd,\n\t\t\t},\n\t\t}\n\n\t\tfor idx, state := range states {\n\t\t\texpect := expected[idx]\n\t\t\trequire.Equal(t, state.Mask, expect.Mask, \"Mask differs\")\n\t\t\trequire.Equal(t, state.Head, expect.Head, \"Head differs\")\n\t\t\trequire.Equal(t, state.Curr, expect.Curr, \"Curr differs\")\n\t\t}\n\t})\n}\n\nfunc TestHistoryWithNoParent(t *testing.T) {\n\tc.WithDummyKv(t, func(kv db.Database) {\n\t\tlkr := c.NewLinker(kv)\n\t\tlkr.SetOwner(\"alice\")\n\n\t\tfile, head := c.MustTouchAndCommit(t, lkr, \"/x\", 1)\n\n\t\thist, err := History(lkr, file, head, nil)\n\t\trequire.Nil(t, err)\n\t\trequire.Len(t, hist, 1)\n\t\trequire.Equal(t, hist[0].Mask, ChangeTypeAdd)\n\t})\n}\n\n// Regression test:\n// Directories loose move history operation\n// when restarting the daemon in between.\nfunc TestHistoryMovedDirsWithReloadedLinker(t *testing.T) {\n\tvalidateHist := func(hist []*Change) {\n\t\trequire.Len(t, hist, 2)\n\t\trequire.Equal(t, hist[0].Mask, ChangeTypeMove)\n\t\trequire.Equal(t, hist[1].Mask, ChangeTypeAdd)\n\t}\n\n\tc.WithReloadingLinker(t, func(lkr *c.Linker) {\n\t\tchildDir := c.MustMkdir(t, lkr, \"/child\")\n\t\tc.MustCommit(t, lkr, \"created\")\n\t\tmovedDir := c.MustMove(t, lkr, childDir, \"/moved_child\")\n\n\t\tstatus, err := lkr.Status()\n\t\trequire.Nil(t, err)\n\n\t\thist, err := History(lkr, movedDir, status, nil)\n\t\trequire.Nil(t, err)\n\n\t\tvalidateHist(hist)\n\t}, func(lkr *c.Linker) {\n\t\tstatus, err := lkr.Status()\n\t\trequire.Nil(t, err)\n\n\t\tchildDir, err := lkr.LookupDirectory(\"/moved_child\")\n\t\trequire.Nil(t, err)\n\n\t\thist, err := History(lkr, childDir, status, nil)\n\t\trequire.Nil(t, err)\n\n\t\tvalidateHist(hist)\n\t})\n}\n\n// Regression test:\nfunc TestHistoryOfMovedNestedDir(t *testing.T) {\n\tc.WithDummyLinker(t, func(lkr *c.Linker) {\n\t\tc.MustMkdir(t, lkr, \"/src/core\")\n\t\tc.MustTouch(t, lkr, \"/src/core/linker.go\", 3)\n\t\tc.MustCommit(t, lkr, \"added\")\n\n\t\tc.MustMove(t, lkr, c.MustLookupDirectory(t, lkr, \"/src\"), \"/dst\")\n\t\tc.MustCommit(t, lkr, \"move\")\n\n\t\tstatus, err := lkr.Status()\n\t\trequire.Nil(t, err)\n\n\t\t// This raised an error before, since \"/dst\" was missing\n\t\t// in the \"added\" commit.\n\t\thist, err := History(lkr, c.MustLookupDirectory(t, lkr, \"/dst/core\"), status, nil)\n\t\trequire.Nil(t, err)\n\n\t\trequire.Equal(t, \"/dst/core\", hist[0].Curr.Path())\n\t\trequire.Equal(t, ChangeTypeNone, hist[0].Mask)\n\t\trequire.Equal(t, \"/dst/core\", hist[1].Curr.Path())\n\t\trequire.Equal(t, ChangeTypeMove, hist[1].Mask)\n\t\trequire.Equal(t, \"/src/core\", hist[1].WasPreviouslyAt)\n\t\trequire.Equal(t, \"/src/core\", hist[2].Curr.Path())\n\t\trequire.Equal(t, ChangeTypeAdd, hist[2].Mask)\n\n\t\tfile, err := lkr.LookupModNode(\"/dst/core/linker.go\")\n\t\trequire.Nil(t, err)\n\n\t\thist, err = History(lkr, file, status, nil)\n\t\trequire.Nil(t, err)\n\n\t\trequire.Equal(t, \"/dst/core/linker.go\", hist[0].Curr.Path())\n\t\trequire.Equal(t, ChangeTypeNone, hist[0].Mask)\n\t\trequire.Equal(t, \"/dst/core/linker.go\", hist[1].Curr.Path())\n\t\trequire.Equal(t, ChangeTypeMove, hist[1].Mask)\n\t\trequire.Equal(t, \"/src/core/linker.go\", hist[1].WasPreviouslyAt)\n\t\trequire.Equal(t, \"/src/core/linker.go\", hist[2].Curr.Path())\n\t\trequire.Equal(t, ChangeTypeAdd, hist[2].Mask)\n\t})\n}\n" }, { "path": "catfs/vcs/mapper.go", "content": "package vcs\n\n// NOTE ON CODING STYLE:\n// If you modify something in here, make sure to always\n// incude \"src\" or \"dst\" in the symbol name to indicate\n// to which side of the sync/diff this symbol belongs!\n// Too many hours have been spent on confused debugging.\n\nimport (\n\t\"fmt\"\n\t\"path\"\n\n\te \"github.com/pkg/errors\"\n\tc \"github.com/sahib/brig/catfs/core\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\t\"github.com/sahib/brig/util/trie\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// MapPair is a pair of nodes (a file or a directory)\n// One of Src and Dst might be nil:\n// - If Src is nil, the node was removed on the remote side.\n// - If Dst is nil, the node was added on the remote side.\n//\n// Both shall never be nil at the same time.\n//\n// If TypeMismatch is true, nodes have a different type\n// and need conflict resolution.\n//\n// If SrcWasRemoved is true, the node was deleted on the\n// remote's side and we might need to propagate this remove.\n// Otherwise, if src is nil, dst can be considered as missing\n// file on src's side.\n//\n// If SrcWasMoved is true, the two nodes were purely moved,\n// but not modified otherwise.\ntype MapPair struct {\n\tSrc n.ModNode\n\tDst n.ModNode\n\n\tSrcWasRemoved bool\n\tSrcWasMoved bool\n\tTypeMismatch bool\n}\n\n// flags that are set during the mapper run.\n// The zero value of this struct should mean \"disabled\".\ntype flags struct {\n\t// The node was visited on the source side.\n\t// This should prohibit duplicate visits.\n\tsrcVisited bool\n\n\t// The file was already reported/tested equal on src side.\n\tsrcHandled bool\n\n\t// The file was already reported/tested equal on dst side.\n\tdstHandled bool\n\n\t// The directory consists completely of other src reports.\n\tsrcComplete bool\n\n\t// The directory consists completely of other dst reports.\n\tdstComplete bool\n}\n\n// Mapper holds the state for the mapping algorithm.\ntype Mapper struct {\n\tlkrSrc, lkrDst *c.Linker\n\tsrcRoot n.Node\n\tsrcHead *n.Commit\n\tdstHead *n.Commit\n\tflagsRoot *trie.Node\n\tfn func(pair MapPair) error\n}\n\nfunc (ma *Mapper) getFlags(path string) *flags {\n\tchild := ma.flagsRoot.Lookup(path)\n\tif child == nil {\n\t\tchild = ma.flagsRoot.InsertWithData(path, &flags{})\n\t}\n\n\tif child.Data == nil {\n\t\tchild.Data = &flags{}\n\t}\n\n\treturn child.Data.(*flags)\n}\n\nfunc (ma *Mapper) setSrcVisited(nd n.Node) {\n\tma.getFlags(nd.Path()).srcVisited = true\n}\n\nfunc (ma *Mapper) setSrcHandled(nd n.Node) {\n\tma.getFlags(nd.Path()).srcHandled = true\n}\n\nfunc (ma *Mapper) setDstHandled(nd n.Node) {\n\tma.getFlags(nd.Path()).dstHandled = true\n}\n\nfunc (ma *Mapper) setSrcComplete(nd n.Node) {\n\tma.getFlags(nd.Path()).srcComplete = true\n}\n\nfunc (ma *Mapper) setDstComplete(nd n.Node) {\n\tma.getFlags(nd.Path()).dstComplete = true\n}\n\nfunc (ma *Mapper) isSrcVisited(nd n.Node) bool {\n\treturn ma.getFlags(nd.Path()).srcVisited\n}\n\nfunc (ma *Mapper) isSrcHandled(nd n.Node) bool {\n\treturn ma.getFlags(nd.Path()).srcHandled\n}\n\nfunc (ma *Mapper) isDstHandled(nd n.Node) bool {\n\treturn ma.getFlags(nd.Path()).dstHandled\n}\n\nfunc (ma *Mapper) isSrcComplete(nd n.Node) bool {\n\treturn ma.getFlags(nd.Path()).srcComplete\n}\n\nfunc (ma *Mapper) isDstComplete(nd n.Node) bool {\n\treturn ma.getFlags(nd.Path()).dstComplete\n}\n\n////////////////////\n\nfunc (ma *Mapper) report(src, dst n.ModNode, typeMismatch, isRemove, isMove bool) error {\n\tif src != nil {\n\t\tma.setSrcHandled(src)\n\t}\n\n\tif dst != nil {\n\t\tma.setDstHandled(dst)\n\t}\n\n\tdebug(\"=> report\", src, dst)\n\treturn ma.fn(MapPair{\n\t\tSrc: src,\n\t\tDst: dst,\n\t\tTypeMismatch: typeMismatch,\n\t\tSrcWasRemoved: isRemove,\n\t\tSrcWasMoved: isMove,\n\t})\n}\n\nfunc (ma *Mapper) reportByType(src, dst n.ModNode) error {\n\tif src == nil || dst == nil {\n\t\treturn ma.report(src, dst, false, false, false)\n\t}\n\n\tisTypeMismatch := src.Type() != dst.Type()\n\tif isTypeMismatch {\n\t\treturn ma.report(src, dst, isTypeMismatch, false, false)\n\t}\n\n\tif src.ContentHash().Equal(dst.ContentHash()) {\n\t\t// If the files are equal, but the location changed,\n\t\t// the file were moved.\n\t\tif src.Path() != dst.Path() {\n\t\t\treturn ma.report(src, dst, isTypeMismatch, false, true)\n\t\t}\n\n\t\t// The files appear to be equal.\n\t\t// We need to remember to not output them again.\n\t\tma.setSrcHandled(src)\n\t\tma.setDstHandled(dst)\n\t\treturn nil\n\t}\n\n\treturn ma.report(src, dst, isTypeMismatch, false, false)\n}\n\nfunc (ma *Mapper) mapFile(srcCurr *n.File, dstFilePath string) error {\n\t// Check if we already visited this file.\n\tif ma.isSrcVisited(srcCurr) {\n\t\treturn nil\n\t}\n\n\tdebug(\"map file\", srcCurr.Path(), dstFilePath)\n\n\t// Remember that we visited this node.\n\tma.setSrcVisited(srcCurr)\n\n\tdstCurr, err := ma.lkrDst.LookupNodeAt(ma.dstHead, dstFilePath)\n\tif err != nil && !ie.IsNoSuchFileError(err) {\n\t\treturn err\n\t}\n\n\tif dstCurr == nil {\n\t\t// We do not have this node yet, mark it for copying.\n\t\treturn ma.report(srcCurr, nil, false, false, false)\n\t}\n\n\tswitch typ := dstCurr.Type(); typ {\n\tcase n.NodeTypeDirectory:\n\t\t// Our node seems to be a directory and theirs a file.\n\t\t// That's not something we can fix.\n\t\tdstDir, ok := dstCurr.(*n.Directory)\n\t\tif !ok {\n\t\t\treturn ie.ErrBadNode\n\t\t}\n\n\t\t// File and Directory don't go well together.\n\t\treturn ma.report(srcCurr, dstDir, true, false, false)\n\tcase n.NodeTypeFile:\n\t\t// We have two competing files.\n\t\tdstFile, ok := dstCurr.(*n.File)\n\t\tif !ok {\n\t\t\treturn ie.ErrBadNode\n\t\t}\n\n\t\treturn ma.reportByType(srcCurr, dstFile)\n\tcase n.NodeTypeGhost:\n\t\t// It's still possible that the file was moved or removed on our side.\n\t\taliveDstCurr, err := ma.ghostToAlive(ma.lkrDst, ma.dstHead, dstCurr)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif aliveDstCurr == nil {\n\t\t\tdstGhost, ok := dstCurr.(*n.Ghost)\n\t\t\tif !ok {\n\t\t\t\treturn ie.ErrBadNode\n\t\t\t}\n\n\t\t\t// File was removed by us.\n\t\t\treturn ma.reportByType(srcCurr, dstGhost)\n\t\t}\n\n\t\treturn ma.reportByType(srcCurr, aliveDstCurr)\n\tdefault:\n\t\treturn e.Wrapf(ie.ErrBadNode, \"Unexpected node type in syncFile: %v\", typ)\n\t}\n}\n\nfunc (ma *Mapper) mapDirectoryContents(srcCurr *n.Directory, dstPath string) error {\n\tsrcChildren, err := srcCurr.ChildrenSorted(ma.lkrSrc)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor _, srcChild := range srcChildren {\n\t\tchildDstPath := path.Join(dstPath, srcChild.Name())\n\t\tswitch srcChild.Type() {\n\t\tcase n.NodeTypeDirectory:\n\t\t\tsrcChildDir, ok := srcChild.(*n.Directory)\n\t\t\tif !ok {\n\t\t\t\treturn ie.ErrBadNode\n\t\t\t}\n\n\t\t\tif err := ma.mapDirectory(srcChildDir, childDstPath, false); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tma.setSrcHandled(srcChildDir)\n\t\t\tdstCurrNd, err := ma.lkrDst.LookupModNodeAt(ma.dstHead, childDstPath)\n\t\t\tif err == nil {\n\t\t\t\tma.setDstHandled(dstCurrNd)\n\t\t\t}\n\t\tcase n.NodeTypeFile:\n\t\t\tsrcChildFile, ok := srcChild.(*n.File)\n\t\t\tif !ok {\n\t\t\t\treturn ie.ErrBadNode\n\t\t\t}\n\n\t\t\tif err := ma.mapFile(srcChildFile, childDstPath); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tma.setSrcHandled(srcChildFile)\n\t\tcase n.NodeTypeGhost:\n\t\t\t// remote ghosts are ignored, since they were handled beforehand.\n\t\tdefault:\n\t\t\treturn ie.ErrBadNode\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (ma *Mapper) mapDirectory(srcCurr *n.Directory, dstPath string, force bool) error {\n\tif !force {\n\t\tif ma.isSrcVisited(srcCurr) {\n\t\t\treturn nil\n\t\t}\n\t}\n\n\tlog.Debugf(\"mapping dir %s <-> %s\", srcCurr.Path(), dstPath)\n\n\tma.setSrcVisited(srcCurr)\n\tdstCurrNd, err := ma.lkrDst.LookupModNodeAt(ma.dstHead, dstPath)\n\tif err != nil && !ie.IsNoSuchFileError(err) {\n\t\treturn err\n\t}\n\n\tif dstCurrNd == nil {\n\t\t// We never heard of this directory apparently. Go sync it.\n\t\treturn ma.report(srcCurr, nil, false, false, false)\n\t}\n\n\t// Special case: The node might have been moved on dst's side.\n\t// We might notice this, if dst type is a ghost.\n\tif dstCurrNd.Type() == n.NodeTypeGhost {\n\t\taliveDstCurr, err := ma.ghostToAlive(ma.lkrDst, ma.dstHead, dstCurrNd)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// No sibling found for this ghost.\n\t\tif aliveDstCurr == nil {\n\t\t\treturn ma.report(srcCurr, nil, false, false, false)\n\t\t}\n\n\t\tlocalBackCheck, err := ma.lkrSrc.LookupNodeAt(ma.srcHead, aliveDstCurr.Path())\n\t\tif err != nil && !ie.IsNoSuchFileError(err) {\n\t\t\treturn err\n\t\t}\n\n\t\tif localBackCheck == nil || localBackCheck.Type() == n.NodeTypeGhost {\n\t\t\t// Delete the guard again, due to the recursive call.\n\t\t\treturn ma.mapDirectory(srcCurr, aliveDstCurr.Path(), true)\n\t\t}\n\n\t\treturn ma.report(srcCurr, nil, false, false, false)\n\t}\n\n\tif dstCurrNd.Type() != n.NodeTypeDirectory {\n\t\treturn ma.report(srcCurr, dstCurrNd, true, false, false)\n\t}\n\n\tdstCurr, ok := dstCurrNd.(*n.Directory)\n\tif !ok {\n\t\treturn ie.ErrBadNode\n\t}\n\n\t// Check if we're lucky and the directory hash is equal:\n\tif srcCurr.ContentHash().Equal(dstCurr.ContentHash()) {\n\t\t// Remember that we visited this subtree.\n\t\tma.setSrcHandled(srcCurr)\n\t\tma.setDstHandled(dstCurr)\n\n\t\tlog.Debugf(\n\t\t\t\"%s and %s have the same content; skipping\",\n\t\t\tsrcCurr.Path(),\n\t\t\tdstCurr.Path(),\n\t\t)\n\n\t\tif srcCurr.Path() != dstCurr.Path() {\n\t\t\treturn ma.report(srcCurr, dstCurr, false, false, true)\n\t\t}\n\n\t\t// If they even have the same tree hash, we can be sure that both\n\t\t// use the same path layout even. No work to do in this case.\n\t\tif srcCurr.TreeHash().Equal(dstCurr.TreeHash()) {\n\t\t\treturn nil\n\t\t}\n\t}\n\n\t// Both sides have this directory, but the content differs.\n\t// We need to figure out recursively what exactly is different.\n\treturn ma.mapDirectoryContents(srcCurr, dstPath)\n}\n\nfunc (ma *Mapper) ghostToAlive(lkr *c.Linker, head *n.Commit, nd n.Node) (n.ModNode, error) {\n\tpartnerNd, _, err := lkr.MoveEntryPoint(nd)\n\tif err != nil {\n\t\treturn nil, e.Wrap(err, \"move entry point\")\n\t}\n\n\t// No move partner found.\n\tif partnerNd == nil {\n\t\treturn nil, nil\n\t}\n\n\t// We want to go forward in history.\n\t// In theory, the other direction should not happen,\n\t// since we're always operating on ghosts here.\n\t// if moveDir != c.MoveDirDstToSrc {\n\t// \tlog.Debugf(\"bad move direction\")\n\t// \treturn nil, nil\n\t// }\n\n\t// Go forward to the most recent version of this node.\n\t// This is no guarantee yet that this node is reachable\n\t// from the head commit (it might have been removed...)\n\tmostRecent, err := lkr.NodeByInode(partnerNd.Inode())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif mostRecent == nil {\n\t\terr = fmt.Errorf(\"mapper: No such node with inode %d\", partnerNd.Inode())\n\t\treturn nil, err\n\t}\n\n\t// This should usually not happen, but just to be sure.\n\tif mostRecent.Type() == n.NodeTypeGhost {\n\t\treturn nil, nil\n\t}\n\n\treacheable, err := lkr.LookupNodeAt(head, mostRecent.Path())\n\tif err != nil && !ie.IsNoSuchFileError(err) {\n\t\treturn nil, e.Wrapf(err, \"ghost2alive: lookupAt\")\n\t}\n\n\tif reacheable == nil {\n\t\treturn nil, nil\n\t}\n\n\tif reacheable.Inode() != mostRecent.Inode() {\n\t\t// The node is still reachable, but it was changed\n\t\t// (i.e. by removing and re-adding it -> different inode)\n\t\treturn nil, nil\n\t}\n\n\treacheableModNd, ok := reacheable.(n.ModNode)\n\tif !ok {\n\t\treturn nil, ie.ErrBadNode\n\t}\n\n\treturn reacheableModNd, nil\n}\n\ntype ghostDir struct {\n\t// source directory.\n\tsrcDir *n.Directory\n\n\t// mapped path in lkrDst\n\tdstPath string\n}\n\nfunc (ma *Mapper) handleGhostsWithoutAliveNd(srcNd n.Node) error {\n\tdstNd, err := ma.lkrDst.LookupNodeAt(ma.dstHead, srcNd.Path())\n\tif err != nil && !ie.IsNoSuchFileError(err) {\n\t\treturn err\n\t}\n\n\t// Check if we maybe already removed or moved the node:\n\tif dstNd != nil && dstNd.Type() != n.NodeTypeGhost {\n\t\tdstModNd, ok := dstNd.(n.ModNode)\n\t\tif !ok {\n\t\t\treturn ie.ErrBadNode\n\t\t}\n\n\t\t// Report that the file is missing on src's side.\n\t\treturn ma.report(nil, dstModNd, false, true, false)\n\t}\n\n\t// does not exist on both sides, nothing to report.\n\treturn nil\n}\n\nfunc (ma *Mapper) extractGhostDirs() ([]ghostDir, error) {\n\tmovedSrcDirs := []ghostDir{}\n\treturn movedSrcDirs, n.Walk(ma.lkrSrc, ma.srcRoot, true, func(srcNd n.Node) error {\n\t\t// Ignore everything that is not a ghost.\n\t\tif srcNd.Type() != n.NodeTypeGhost {\n\t\t\treturn nil\n\t\t}\n\n\t\taliveSrcNd, err := ma.ghostToAlive(ma.lkrSrc, ma.srcHead, srcNd)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif aliveSrcNd == nil {\n\t\t\t// It's a ghost, but it has no living counterpart.\n\t\t\t// This node *might* have been removed on the remote side.\n\t\t\t// Try to see if we have a node at this path, the next step\n\t\t\t// of sync then needs to decide if the node needs to be removed.\n\t\t\treturn ma.handleGhostsWithoutAliveNd(srcNd)\n\t\t}\n\n\t\t// At this point we know that the ghost related to a moved file.\n\t\t// Check if we have a file at the same place.\n\t\tdstNd, err := ma.lkrDst.LookupNodeAt(ma.dstHead, aliveSrcNd.Path())\n\t\tif err != nil && !ie.IsNoSuchFileError(err) {\n\t\t\treturn err\n\t\t}\n\n\t\tif dstNd != nil && dstNd.Type() != n.NodeTypeGhost {\n\t\t\t// The node already exists in our place. No way we can really merge\n\t\t\t// it cleanly, so just handle the ghost as normal file and potentially\n\t\t\t// apply the normal conflict resolution later on.\n\t\t\treturn nil\n\t\t}\n\n\t\tdstRefNd, err := ma.lkrDst.LookupNodeAt(ma.dstHead, srcNd.Path())\n\t\tif err != nil && !ie.IsNoSuchFileError(err) {\n\t\t\treturn err\n\t\t}\n\n\t\tif dstRefNd != nil {\n\t\t\t// Node maybe also moved. If so, try to resolve it to the full node:\n\t\t\tif dstRefNd.Type() == n.NodeTypeGhost {\n\t\t\t\taliveOrig, err := ma.ghostToAlive(ma.lkrDst, ma.dstHead, dstRefNd)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\n\t\t\t\tdstRefNd = aliveOrig\n\t\t\t}\n\t\t}\n\n\t\t// The node was removed on dst:\n\t\t// We will detect the removal later.\n\t\tif dstRefNd == nil {\n\t\t\treturn nil\n\t\t}\n\n\t\tdstRefModNd, ok := dstRefNd.(n.ModNode)\n\t\tif !ok {\n\t\t\treturn e.Wrapf(ie.ErrBadNode, \"dstRefModNd is not a file or directory: %v\", dstRefNd)\n\t\t}\n\n\t\tswitch aliveSrcNd.Type() {\n\t\tcase n.NodeTypeFile:\n\t\t\t// Mark those both ghosts and original node as visited.\n\t\t\terr = ma.mapFile(aliveSrcNd.(*n.File), dstRefModNd.Path())\n\t\t\tma.setSrcVisited(aliveSrcNd)\n\t\t\tma.setSrcVisited(srcNd)\n\t\t\treturn err\n\t\tcase n.NodeTypeDirectory:\n\t\t\t// ma.setSrcVisited(srcNd)\n\t\t\tif dstRefNd.Type() != n.NodeTypeDirectory {\n\t\t\t\treturn ma.report(aliveSrcNd, dstRefModNd, true, false, false)\n\t\t\t}\n\n\t\t\taliveSrcDir, ok := aliveSrcNd.(*n.Directory)\n\t\t\tif !ok {\n\t\t\t\treturn ie.ErrBadNode\n\t\t\t}\n\n\t\t\tmovedSrcDirs = append(movedSrcDirs, ghostDir{\n\t\t\t\tsrcDir: aliveSrcDir,\n\t\t\t\tdstPath: dstRefNd.Path(),\n\t\t\t})\n\n\t\t\treturn nil\n\t\tdefault:\n\t\t\treturn e.Wrapf(ie.ErrBadNode, \"Unexpected type in handle ghosts: %v\", err)\n\t\t}\n\t})\n}\n\nfunc (ma *Mapper) handleGhosts() error {\n\tmovedSrcDirs, err := ma.extractGhostDirs()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Handle moved paths after handling single files.\n\t// (mapDirectory assumes that moved files in it were already handled).\n\tfor _, movedSrcDir := range movedSrcDirs {\n\t\tlog.Debugf(\"map: %v %v\", movedSrcDir.srcDir.Path(), movedSrcDir.dstPath)\n\t\tif err := ma.mapDirectory(movedSrcDir.srcDir, movedSrcDir.dstPath, false); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// NewMapper creates a new mapper object that is capable of finding pairs of\n// nodes between lkrDst and lkrSrc.\nfunc NewMapper(lkrSrc, lkrDst *c.Linker, srcHead, dstHead *n.Commit, srcRoot n.Node) (*Mapper, error) {\n\tvar err error\n\n\tif srcHead == nil {\n\t\tsrcHead, err = lkrSrc.Head()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tif dstHead == nil {\n\t\tdstHead, err = lkrDst.Head()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn &Mapper{\n\t\tlkrSrc: lkrSrc,\n\t\tlkrDst: lkrDst,\n\t\tsrcHead: srcHead,\n\t\tdstHead: dstHead,\n\t\tsrcRoot: srcRoot,\n\t\tflagsRoot: trie.NewNodeWithData(&flags{}),\n\t}, nil\n}\n\nfunc (ma *Mapper) nodeIsHandled(nd n.Node, srcToDst bool) bool {\n\tif srcToDst {\n\t\treturn ma.isSrcHandled(nd)\n\t}\n\n\treturn ma.isDstHandled(nd)\n}\n\nfunc (ma *Mapper) isComplete(lkr *c.Linker, root n.Node, srcToDst bool) (bool, error) {\n\t// If the file was already handled: ignore it completely.\n\tif ma.nodeIsHandled(root, srcToDst) {\n\t\treturn false, nil\n\t}\n\n\tif root.Type() != n.NodeTypeDirectory {\n\t\treturn true, nil\n\t}\n\n\tdir, ok := root.(*n.Directory)\n\tif !ok {\n\t\treturn false, ie.ErrBadNode\n\t}\n\n\tchildren, err := dir.ChildrenSorted(lkr)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\tnComplete := 0\n\tfor _, child := range children {\n\t\tif ma.nodeIsHandled(child, srcToDst) {\n\t\t\tcontinue\n\t\t}\n\n\t\tisComplete, err := ma.isComplete(lkr, child, srcToDst)\n\t\tif err != nil {\n\t\t\treturn false, err\n\t\t}\n\n\t\tif isComplete {\n\t\t\tnComplete++\n\t\t}\n\t}\n\n\t// If all children were not handled & are complete we copy the flag.\n\tif nComplete == len(children) {\n\t\tif srcToDst {\n\t\t\tma.setSrcComplete(root)\n\t\t} else {\n\t\t\tma.setDstComplete(root)\n\t\t}\n\n\t\treturn true, nil\n\t}\n\n\treturn false, nil\n}\n\n// extractLeftovers goes over all nodes in src that were not covered\n// yet by previous measures. It will report any src node without a match then.\nfunc (ma *Mapper) extractLeftovers(lkr *c.Linker, root *n.Directory, srcToDst bool) error {\n\tif ma.nodeIsHandled(root, srcToDst) {\n\t\treturn nil\n\t}\n\n\tif _, err := ma.isComplete(lkr, root, srcToDst); err != nil {\n\t\treturn err\n\t}\n\n\t// Implement a basic walk/DFS with filtering:\n\tchildren, err := root.ChildrenSorted(lkr)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor _, child := range children {\n\t\tdebug(fmt.Sprintf(\"extract: %v\", child.Path()))\n\t\tif ma.nodeIsHandled(child, srcToDst) {\n\t\t\tdebug(fmt.Sprintf(\"node is handled: %v\", child.Path()))\n\t\t\tcontinue\n\t\t}\n\n\t\tswitch child.Type() {\n\t\tcase n.NodeTypeDirectory:\n\t\t\tdir, ok := child.(*n.Directory)\n\t\t\tif !ok {\n\t\t\t\treturn ie.ErrBadNode\n\t\t\t}\n\n\t\t\tvar complete bool\n\t\t\tif srcToDst {\n\t\t\t\tcomplete = ma.isSrcComplete(dir)\n\t\t\t} else {\n\t\t\t\tcomplete = ma.isDstComplete(dir)\n\t\t\t}\n\n\t\t\tdebug(fmt.Sprintf(\"is complete: %v %v\", child.Path(), complete))\n\t\t\tif complete {\n\t\t\t\tif srcToDst {\n\t\t\t\t\terr = ma.report(dir, nil, false, false, false)\n\t\t\t\t} else {\n\t\t\t\t\terr = ma.report(nil, dir, false, false, false)\n\t\t\t\t}\n\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif err := ma.extractLeftovers(lkr, dir, srcToDst); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\t\tcase n.NodeTypeFile:\n\t\t\tfile, ok := child.(*n.File)\n\t\t\tif !ok {\n\t\t\t\treturn ie.ErrBadNode\n\t\t\t}\n\n\t\t\t// Report the leftover:\n\t\t\tif srcToDst {\n\t\t\t\terr = ma.report(file, nil, false, false, false)\n\t\t\t} else {\n\t\t\t\terr = ma.report(nil, file, false, false, false)\n\t\t\t}\n\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\tcase n.NodeTypeGhost:\n\t\t\t// Those were already handled (or are not important)\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// Map calls `fn` for each pairing that was found. Equal files and\n// directories are not reported. Most directories are also not reported, but\n// if they are empty and not present on our side they will. No ghosts will be\n// reported.\n//\n// Some implementation background for the curious reader:\n//\n// In the simplest case a filesystem is a tree and the assumption can be made\n// that one node that lives at the same path on both sides is the same \"file\"\n// (i.e. in terms of \"this is the file that the user wants to synchronize with\").\n//\n// With ghosts though, we have nodes that can indicate a removed or a moved file.\n// Due to moved files the filesystem tree becomes a graph and the mapping\n// algorithm (that is the base of Mapper) needs to do a depth first search\n// and thus needs to remember already visited nodes.\n//\n// Since moved nodes also takes priority we need to iterate over all ghosts first,\n// and mark their respective counterparts or report that they were removed on\n// the remote side (i.e. no counterpart exists.). Only after that we cycle\n// through all other nodes and assume that files living at the same path\n// reference the same \"file\". At this point we can treat the file graph\n// as tree again by ignoring all ghosts.\n//\n// A special case is when a file was moved on one side but, a file exists\n// already on the other side. In this case the already existing files wins.\n//\n// Some examples of the described behaviours can be found in the tests of Mapper.\nfunc (ma *Mapper) Map(fn func(pair MapPair) error) error {\n\tma.fn = fn\n\tlog.Debugf(\"mapping ghosts\")\n\tif err := ma.handleGhosts(); err != nil {\n\t\treturn err\n\t}\n\n\tlog.Debugf(\"mapping non-ghosts\")\n\n\tswitch ma.srcRoot.Type() {\n\tcase n.NodeTypeDirectory:\n\t\tdir, ok := ma.srcRoot.(*n.Directory)\n\t\tif !ok {\n\t\t\treturn ie.ErrBadNode\n\t\t}\n\n\t\tif err := ma.mapDirectory(dir, dir.Path(), false); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Get root directories:\n\t\t// (only get them now since, in theory, mapFn could have changed things)\n\t\tsrcRoot, err := ma.lkrSrc.DirectoryByHash(ma.srcHead.Root())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tdstRoot, err := ma.lkrDst.DirectoryByHash(ma.dstHead.Root())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdebug(\"-- Extract leftover src\")\n\n\t\t// Extract things in \"src\" that were not mapped yet.\n\t\t// These are files that can be added to our inventory,\n\t\t// since we have notthing that mapped to them.\n\t\tif err := ma.extractLeftovers(ma.lkrSrc, srcRoot, true); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdebug(\"-- Extract leftover dst\")\n\n\t\t// Check for files that we have, but dst does not.\n\t\t// We call those files \"missing\".\n\t\treturn ma.extractLeftovers(ma.lkrDst, dstRoot, false)\n\tcase n.NodeTypeFile:\n\t\tfile, ok := ma.srcRoot.(*n.File)\n\t\tif !ok {\n\t\t\treturn ie.ErrBadNode\n\t\t}\n\n\t\treturn ma.mapFile(file, file.Path())\n\tcase n.NodeTypeGhost:\n\t\t// Not sure how this would happen.\n\t\treturn nil\n\tdefault:\n\t\treturn e.Wrapf(ie.ErrBadNode, \"Unexpected type in route(): %v\", ma.srcRoot)\n\t}\n}\n" }, { "path": "catfs/vcs/mapper_test.go", "content": "package vcs\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n\n\tc \"github.com/sahib/brig/catfs/core\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc mapperSetupBasicSame(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tc.MustTouchAndCommit(t, lkrSrc, \"/x.png\", 23)\n\tc.MustTouchAndCommit(t, lkrDst, \"/x.png\", 23)\n\treturn []MapPair{}\n}\n\nfunc mapperSetupBasicDiff(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tsrcFile, _ := c.MustTouchAndCommit(t, lkrSrc, \"/x.png\", 23)\n\tdstFile, _ := c.MustTouchAndCommit(t, lkrDst, \"/x.png\", 42)\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: srcFile,\n\t\t\tDst: dstFile,\n\t\t\tTypeMismatch: false,\n\t\t},\n\t}\n}\n\nfunc mapperSetupBasicSrcTypeMismatch(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tsrcDir := c.MustMkdir(t, lkrSrc, \"/x\")\n\tc.MustCommit(t, lkrSrc, \"add dir\")\n\n\tdstFile, _ := c.MustTouchAndCommit(t, lkrDst, \"/x\", 42)\n\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: srcDir,\n\t\t\tDst: dstFile,\n\t\t\tTypeMismatch: true,\n\t\t},\n\t}\n}\n\nfunc mapperSetupBasicDstTypeMismatch(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tsrcFile, _ := c.MustTouchAndCommit(t, lkrSrc, \"/x\", 42)\n\tdstDir := c.MustMkdir(t, lkrDst, \"/x\")\n\tc.MustCommit(t, lkrDst, \"add dir\")\n\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: srcFile,\n\t\t\tDst: dstDir,\n\t\t\tTypeMismatch: true,\n\t\t},\n\t}\n}\n\nfunc mapperSetupBasicSrcAddFile(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tsrcFile, _ := c.MustTouchAndCommit(t, lkrSrc, \"/x.png\", 42)\n\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: srcFile,\n\t\t\tDst: nil,\n\t\t\tTypeMismatch: false,\n\t\t},\n\t}\n}\n\nfunc mapperSetupBasicDstAddFile(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tdstFile, _ := c.MustTouchAndCommit(t, lkrDst, \"/x.png\", 42)\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: nil,\n\t\t\tDst: dstFile,\n\t\t\tTypeMismatch: false,\n\t\t},\n\t}\n}\n\nfunc mapperSetupBasicSrcAddDir(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tsrcDir := c.MustMkdir(t, lkrSrc, \"/x\")\n\tc.MustCommit(t, lkrSrc, \"add dir\")\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: srcDir,\n\t\t\tDst: nil,\n\t\t\tTypeMismatch: false,\n\t\t},\n\t}\n}\n\nfunc mapperSetupBasicDstAddDir(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tc.MustMkdir(t, lkrDst, \"/x\")\n\treturn []MapPair{}\n}\n\nfunc mapperSetupSrcMoveFile(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tdstFile, _ := c.MustTouchAndCommit(t, lkrDst, \"/x.png\", 42)\n\tsrcFileOld, _ := c.MustTouchAndCommit(t, lkrSrc, \"/x.png\", 23)\n\tsrcFile := c.MustMove(t, lkrSrc, srcFileOld, \"/y.png\")\n\tc.MustCommit(t, lkrSrc, \"I like to move it\")\n\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: srcFile,\n\t\t\tDst: dstFile,\n\t\t\tTypeMismatch: false,\n\t\t},\n\t}\n}\n\nfunc mapperSetupDstMoveFile(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tsrcFile, _ := c.MustTouchAndCommit(t, lkrSrc, \"/x.png\", 42)\n\tdstFileOld, _ := c.MustTouchAndCommit(t, lkrDst, \"/x.png\", 23)\n\tdstFile := c.MustMove(t, lkrDst, dstFileOld, \"/y.png\")\n\tc.MustCommit(t, lkrDst, \"I like to move it, move it\")\n\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: srcFile,\n\t\t\tDst: dstFile,\n\t\t\tTypeMismatch: false,\n\t\t},\n\t}\n}\n\nfunc mapperMoveNestedDir(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tc.MustMkdir(t, lkrSrc, \"/old/sub/\")\n\tc.MustMkdir(t, lkrDst, \"/old/sub/\")\n\tc.MustTouchAndCommit(t, lkrSrc, \"/old/sub/x\", 1)\n\tc.MustTouchAndCommit(t, lkrDst, \"/old/sub/x\", 1)\n\n\tsrcDir := c.MustLookupDirectory(t, lkrSrc, \"/old\")\n\tdstDir := c.MustLookupDirectory(t, lkrDst, \"/old\")\n\tnewDstDir := c.MustMove(t, lkrDst, dstDir, \"/new\")\n\tc.MustCommit(t, lkrDst, \"moved\")\n\n\t// Test for a special case here:\n\t// Directories that were moved, but still have identical files.\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: srcDir,\n\t\t\tDst: newDstDir,\n\t\t\tSrcWasMoved: true,\n\t\t\tTypeMismatch: false,\n\t\t\tSrcWasRemoved: false,\n\t\t},\n\t}\n}\n\nfunc mapperSetupDstMoveDirEmpty(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tsrcDir := c.MustMkdir(t, lkrSrc, \"/x\")\n\tc.MustCommit(t, lkrSrc, \"Create src dir\")\n\n\tdstDirOld := c.MustMkdir(t, lkrDst, \"/x\")\n\tdstDir := c.MustMove(t, lkrDst, dstDirOld, \"/y\")\n\tc.MustCommit(t, lkrDst, \"I like to move it, move it\")\n\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: srcDir,\n\t\t\tDst: dstDir,\n\t\t\tSrcWasMoved: true,\n\t\t},\n\t}\n}\n\nfunc mapperSetupDstMoveDir(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tc.MustMkdir(t, lkrSrc, \"/x\")\n\tsrcFile := c.MustTouch(t, lkrSrc, \"/x/a.png\", 42)\n\tc.MustCommit(t, lkrSrc, \"Create src dir\")\n\n\tdstDirOld := c.MustMkdir(t, lkrDst, \"/x\")\n\tc.MustMove(t, lkrDst, dstDirOld, \"/y\")\n\tdstFile := c.MustTouch(t, lkrDst, \"/y/a.png\", 23)\n\tc.MustCommit(t, lkrDst, \"I like to move it, move it\")\n\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: srcFile,\n\t\t\tDst: dstFile,\n\t\t\tTypeMismatch: false,\n\t\t},\n\t}\n}\n\nfunc mapperSetupSrcMoveDir(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tsrcDirOld := c.MustMkdir(t, lkrSrc, \"/x\")\n\tc.MustMove(t, lkrSrc, srcDirOld, \"/y\")\n\tsrcFile := c.MustTouch(t, lkrSrc, \"/y/a.png\", 23)\n\tc.MustCommit(t, lkrSrc, \"I like to move it, move it\")\n\n\tc.MustMkdir(t, lkrDst, \"/x\")\n\tdstFile := c.MustTouch(t, lkrDst, \"/x/a.png\", 42)\n\tc.MustCommit(t, lkrDst, \"Create dst dir\")\n\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: srcFile,\n\t\t\tDst: dstFile,\n\t\t\tTypeMismatch: false,\n\t\t},\n\t}\n}\n\nfunc mapperSetupMoveDirWithChild(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tsrcDirOld := c.MustMkdir(t, lkrSrc, \"/x\")\n\tsrcFile := c.MustTouch(t, lkrSrc, \"/x/a.png\", 23)\n\tc.MustMove(t, lkrSrc, srcDirOld, \"/y\")\n\tc.MustCommit(t, lkrSrc, \"I like to move it, move it\")\n\n\tc.MustMkdir(t, lkrDst, \"/x\")\n\tdstFile := c.MustTouch(t, lkrDst, \"/x/a.png\", 42)\n\tc.MustCommit(t, lkrDst, \"Create dst dir\")\n\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: srcFile,\n\t\t\tDst: dstFile,\n\t\t\tTypeMismatch: false,\n\t\t},\n\t}\n}\n\nfunc mapperSetupSrcMoveWithExisting(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tsrcDirOld := c.MustMkdir(t, lkrSrc, \"/x\")\n\tc.MustMove(t, lkrSrc, srcDirOld, \"/y\")\n\tsrcFile := c.MustTouch(t, lkrSrc, \"/y/a.png\", 23)\n\tc.MustCommit(t, lkrSrc, \"I like to move it, move it\")\n\n\t// Additionally create an existing file that lives in the place\n\t// of the moved file. Mapper should favour existing files:\n\tdstDir := c.MustMkdir(t, lkrDst, \"/x\")\n\tc.MustMkdir(t, lkrDst, \"/y\")\n\tc.MustTouch(t, lkrDst, \"/x/a.png\", 42)\n\tdstFile := c.MustTouch(t, lkrDst, \"/y/a.png\", 42)\n\tc.MustCommit(t, lkrDst, \"Create src dir\")\n\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: srcFile,\n\t\t\tDst: dstFile,\n\t\t\tTypeMismatch: false,\n\t\t}, {\n\t\t\tSrc: nil,\n\t\t\tDst: dstDir,\n\t\t\tTypeMismatch: false,\n\t\t},\n\t}\n}\n\nfunc mapperSetupSrcFileMoveToExistingEmptyDir(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tc.MustMkdir(t, lkrSrc, \"/d1\")\n\tc.MustMkdir(t, lkrSrc, \"/d2\")\n\tsrcFileOld, _ := c.MustTouchAndCommit(t, lkrSrc, \"/d1/t1\", 23)\n\tsrcFile := c.MustMove(t, lkrSrc, srcFileOld, \"/d2/t1\")\n\tc.MustCommit(t, lkrSrc, \"move is done\")\n\n\tc.MustMkdir(t, lkrDst, \"/d1\")\n\tc.MustMkdir(t, lkrDst, \"/d2\")\n\tdstFile, _ := c.MustTouchAndCommit(t, lkrDst, \"/d1/t1\", 23)\n\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: srcFile,\n\t\t\tDst: dstFile,\n\t\t\tSrcWasMoved: true,\n\t\t},\n\t}\n}\n\nfunc mapperSetupDstMoveWithExisting(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tsrcDir := c.MustMkdir(t, lkrSrc, \"/x\")\n\tc.MustMkdir(t, lkrSrc, \"/y\")\n\tc.MustTouch(t, lkrSrc, \"/x/a.png\", 42)\n\tsrcFile := c.MustTouch(t, lkrSrc, \"/y/a.png\", 42)\n\tc.MustCommit(t, lkrSrc, \"Create src dir\")\n\n\tdstDirOld := c.MustMkdir(t, lkrDst, \"/x\")\n\tc.MustMove(t, lkrDst, dstDirOld, \"/y\")\n\tdstFile := c.MustTouch(t, lkrDst, \"/y/a.png\", 23)\n\tc.MustCommit(t, lkrDst, \"I like to move it, move it\")\n\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: srcDir,\n\t\t\tDst: nil,\n\t\t\tTypeMismatch: false,\n\t\t}, {\n\t\t\tSrc: srcFile,\n\t\t\tDst: dstFile,\n\t\t\tTypeMismatch: false,\n\t\t},\n\t}\n}\n\nfunc mapperSetupNested(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tsrcX, _ := c.MustTouchAndCommit(t, lkrSrc, \"/common/a/b/c/x\", 42)\n\tsrcY, _ := c.MustTouchAndCommit(t, lkrSrc, \"/common/a/b/c/y\", 23)\n\tsrcZ, _ := c.MustTouchAndCommit(t, lkrSrc, \"/src-only/z\", 23)\n\n\tdstX, _ := c.MustTouchAndCommit(t, lkrDst, \"/common/a/b/c/x\", 43)\n\tdstY, _ := c.MustTouchAndCommit(t, lkrDst, \"/common/a/b/c/y\", 24)\n\tdstZ, _ := c.MustTouchAndCommit(t, lkrDst, \"/dst-only/z\", 23)\n\n\tsrcZParent, err := n.ParentDirectory(lkrSrc, srcZ)\n\tif err != nil {\n\t\tt.Fatalf(\"setup failed to get parent dir: %v\", err)\n\t}\n\n\tdstZParent, err := n.ParentDirectory(lkrDst, dstZ)\n\tif err != nil {\n\t\tt.Fatalf(\"setup failed to get parent dir: %v\", err)\n\t}\n\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: srcX,\n\t\t\tDst: dstX,\n\t\t\tTypeMismatch: false,\n\t\t}, {\n\t\t\tSrc: srcY,\n\t\t\tDst: dstY,\n\t\t\tTypeMismatch: false,\n\t\t}, {\n\t\t\tSrc: srcZParent,\n\t\t\tDst: nil,\n\t\t\tTypeMismatch: false,\n\t\t}, {\n\t\t\tSrc: nil,\n\t\t\tDst: dstZParent,\n\t\t\tTypeMismatch: false,\n\t\t},\n\t}\n}\n\nfunc mapperSetupSrcRemove(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tsrcFile := c.MustTouch(t, lkrSrc, \"/x.png\", 23)\n\tc.MustCommit(t, lkrSrc, \"src: Touched /x.png\")\n\tc.MustRemove(t, lkrSrc, srcFile)\n\tc.MustCommit(t, lkrSrc, \"src: Removed /x.png\")\n\n\tdstFile := c.MustTouch(t, lkrDst, \"/x.png\", 23)\n\tc.MustCommit(t, lkrDst, \"dst: Touched /x.png\")\n\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: nil,\n\t\t\tDst: dstFile,\n\t\t\tTypeMismatch: false,\n\t\t\tSrcWasRemoved: true,\n\t\t},\n\t}\n}\n\nfunc mapperSetupDstRemove(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tsrcFile := c.MustTouch(t, lkrSrc, \"/x.png\", 23)\n\tc.MustCommit(t, lkrSrc, \"dst: Touched /x.png\")\n\n\tdstFile := c.MustTouch(t, lkrDst, \"/x.png\", 23)\n\tc.MustCommit(t, lkrDst, \"src: Touched /x.png\")\n\tc.MustRemove(t, lkrDst, dstFile)\n\tc.MustCommit(t, lkrDst, \"src: Removed /x.png\")\n\tdstGhost, err := lkrDst.LookupGhost(\"/x.png\")\n\trequire.NoError(t, err)\n\n\t// We should be notified remote removed the file\n\t// (and that we possibly should remove it as well)\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: srcFile,\n\t\t\tDst: dstGhost,\n\t\t\tTypeMismatch: true,\n\t\t},\n\t}\n}\n\nfunc mapperSetupMoveOnBothSides(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair {\n\tsrcFile := c.MustTouch(t, lkrSrc, \"/x\", 23)\n\tc.MustCommit(t, lkrSrc, \"src: touched /x\")\n\tsrcFileMoved := c.MustMove(t, lkrSrc, srcFile, \"/y\")\n\tc.MustCommit(t, lkrSrc, \"src: /x moved to /y\")\n\n\tdstFile := c.MustTouch(t, lkrDst, \"/x\", 42)\n\tc.MustCommit(t, lkrDst, \"dst: touched /x\")\n\tdstFileMoved := c.MustMove(t, lkrDst, dstFile, \"/z\")\n\tc.MustCommit(t, lkrDst, \"dst: /x moved to /z\")\n\n\treturn []MapPair{\n\t\t{\n\t\t\tSrc: srcFileMoved,\n\t\t\tDst: dstFileMoved,\n\t\t\tTypeMismatch: false,\n\t\t},\n\t}\n}\n\nfunc TestMapper(t *testing.T) {\n\ttcs := []struct {\n\t\tname string\n\t\tsetup func(t *testing.T, lkrSrc, lkrDst *c.Linker) []MapPair\n\t}{\n\t\t{\n\t\t\tname: \"basic-same\",\n\t\t\tsetup: mapperSetupBasicSame,\n\t\t}, {\n\t\t\tname: \"basic-diff\",\n\t\t\tsetup: mapperSetupBasicDiff,\n\t\t}, {\n\t\t\tname: \"basic-src-add-file\",\n\t\t\tsetup: mapperSetupBasicSrcAddFile,\n\t\t}, {\n\t\t\tname: \"basic-dst-add-file\",\n\t\t\tsetup: mapperSetupBasicDstAddFile,\n\t\t}, {\n\t\t\tname: \"basic-src-add-dir\",\n\t\t\tsetup: mapperSetupBasicSrcAddDir,\n\t\t}, {\n\t\t\tname: \"basic-dst-add-dir\",\n\t\t\tsetup: mapperSetupBasicDstAddDir,\n\t\t}, {\n\t\t\tname: \"basic-src-type-mismatch\",\n\t\t\tsetup: mapperSetupBasicSrcTypeMismatch,\n\t\t}, {\n\t\t\tname: \"basic-dst-type-mismatch\",\n\t\t\tsetup: mapperSetupBasicDstTypeMismatch,\n\t\t}, {\n\t\t\tname: \"basic-nested\",\n\t\t\tsetup: mapperSetupNested,\n\t\t}, {\n\t\t\tname: \"remove-src\",\n\t\t\tsetup: mapperSetupSrcRemove,\n\t\t}, {\n\t\t\tname: \"remove-dst\",\n\t\t\tsetup: mapperSetupDstRemove,\n\t\t}, {\n\t\t\tname: \"move-simple-src-file\",\n\t\t\tsetup: mapperSetupSrcMoveFile,\n\t\t}, {\n\t\t\tname: \"move-simple-dst-file\",\n\t\t\tsetup: mapperSetupDstMoveFile,\n\t\t}, {\n\t\t\tname: \"move-simple-dst-empty-dir\",\n\t\t\tsetup: mapperSetupDstMoveDirEmpty,\n\t\t}, {\n\t\t\tname: \"move-simple-src-dir\",\n\t\t\tsetup: mapperSetupSrcMoveDir,\n\t\t}, {\n\t\t\tname: \"move-simple-dst-dir\",\n\t\t\tsetup: mapperSetupDstMoveDir,\n\t\t}, {\n\t\t\tname: \"move-simple-src-dir-with-existing\",\n\t\t\tsetup: mapperSetupSrcMoveWithExisting,\n\t\t}, {\n\t\t\tname: \"move-simple-dst-dir-with-existing\",\n\t\t\tsetup: mapperSetupDstMoveWithExisting,\n\t\t}, {\n\t\t\tname: \"move-on-both-sides\",\n\t\t\tsetup: mapperSetupMoveOnBothSides,\n\t\t}, {\n\t\t\tname: \"move-dir-with-child\",\n\t\t\tsetup: mapperSetupMoveDirWithChild,\n\t\t}, {\n\t\t\tname: \"move-nested-dir\",\n\t\t\tsetup: mapperMoveNestedDir,\n\t\t}, {\n\t\t\tname: \"move-src-file-to-existing-empty-dir\",\n\t\t\tsetup: mapperSetupSrcFileMoveToExistingEmptyDir,\n\t\t},\n\t}\n\n\tfor _, tc := range tcs {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tc.WithLinkerPair(t, func(lkrSrc, lkrDst *c.Linker) {\n\t\t\t\texpect := tc.setup(t, lkrSrc, lkrDst)\n\n\t\t\t\tsrcRoot, err := lkrSrc.Root()\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Fatalf(\"Failed to retrieve root: %v\", err)\n\t\t\t\t}\n\n\t\t\t\tgot := []MapPair{}\n\t\t\t\tdiffFn := func(pair MapPair) error {\n\t\t\t\t\tgot = append(got, pair)\n\t\t\t\t\t// if pair.Src != nil {\n\t\t\t\t\t// \tfmt.Println(\".. \", pair.Src.Path())\n\t\t\t\t\t// }\n\t\t\t\t\t// if pair.Dst != nil {\n\t\t\t\t\t// \tfmt.Println(\"-> \", pair.Dst.Path())\n\t\t\t\t\t// }\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\n\t\t\t\tmapper, err := NewMapper(lkrSrc, lkrDst, nil, nil, srcRoot)\n\t\t\t\trequire.Nil(t, err)\n\n\t\t\t\tif err := mapper.Map(diffFn); err != nil {\n\t\t\t\t\tt.Fatalf(\"mapping failed: %v\", err)\n\t\t\t\t}\n\n\t\t\t\t// DEBUG.\n\t\t\t\t// for _, pair := range got {\n\t\t\t\t// \tfmt.Println(\"-\", pair.Src, pair.Dst)\n\t\t\t\t// }\n\n\t\t\t\tif len(got) != len(expect) {\n\t\t\t\t\tt.Fatalf(\n\t\t\t\t\t\t\"Got and expect length differ: %d vs %d\",\n\t\t\t\t\t\tlen(got), len(expect),\n\t\t\t\t\t)\n\t\t\t\t}\n\n\t\t\t\tfor idx, gotPair := range got {\n\t\t\t\t\texpectPair := expect[idx]\n\t\t\t\t\tfailMsg := fmt.Sprintf(\"Failed pair %d\", idx+1)\n\t\t\t\t\trequire.Equal(t, expectPair, gotPair, failMsg)\n\t\t\t\t}\n\t\t\t})\n\t\t})\n\t}\n}\n" }, { "path": "catfs/vcs/patch.go", "content": "package vcs\n\nimport (\n\t\"errors\"\n\t\"path\"\n\t\"sort\"\n\n\te \"github.com/pkg/errors\"\n\tc \"github.com/sahib/brig/catfs/core\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\tcapnp_patch \"github.com/sahib/brig/catfs/vcs/capnp\"\n\t\"github.com/sahib/brig/util/trie\"\n\tlog \"github.com/sirupsen/logrus\"\n\tcapnp \"zombiezen.com/go/capnproto2\"\n)\n\n// Patch is a set of changes that changed since a certain\n// version of a graph.\ntype Patch struct {\n\tFromIndex int64\n\tCurrIndex int64\n\tChanges []*Change\n}\n\n// Patches is just a list of patches\ntype Patches []*Patch\n\n// Len returns the number of changes in the patch.\nfunc (p *Patch) Len() int {\n\treturn len(p.Changes)\n}\n\nfunc (p *Patch) Swap(i, j int) {\n\tp.Changes[i], p.Changes[j] = p.Changes[j], p.Changes[i]\n}\n\nfunc (p *Patch) Less(i, j int) bool {\n\tna, nb := p.Changes[i].Curr, p.Changes[j].Curr\n\n\tnaIsGhost := na.Type() == n.NodeTypeGhost\n\tnbIsGhost := nb.Type() == n.NodeTypeGhost\n\tif naIsGhost != nbIsGhost {\n\t\t// Make sure ghosts are first added\n\t\treturn naIsGhost\n\t}\n\n\tnaIsDir := na.Type() == n.NodeTypeDirectory\n\tnbIsDir := nb.Type() == n.NodeTypeDirectory\n\tif naIsDir != nbIsDir {\n\t\t// Make sure that we first apply directory creation\n\t\t// and possible directory moves.\n\t\treturn naIsDir\n\t}\n\n\tnaIsRemove := p.Changes[i].Mask&ChangeTypeRemove != 0\n\tnbIsRemove := p.Changes[j].Mask&ChangeTypeRemove != 0\n\tif naIsRemove != nbIsRemove {\n\t\t// Make sure that everything is removed before\n\t\t// doing any other changes.\n\t\treturn naIsRemove\n\t}\n\n\tnaIsMove := p.Changes[i].Mask&ChangeTypeMove != 0\n\tnbIsMove := p.Changes[j].Mask&ChangeTypeMove != 0\n\tif naIsMove != nbIsMove {\n\t\t// Make sure that everything is moved before\n\t\t// doing any adds / modifcations.\n\t\treturn naIsMove\n\t}\n\n\treturn na.ModTime().Before(nb.ModTime())\n}\n\nfunc (p *Patch) toCapnpPatch(seg *capnp.Segment, capPatch capnp_patch.Patch) error {\n\tcapPatch.SetFromIndex(p.FromIndex)\n\tcapPatch.SetCurrIndex(p.CurrIndex)\n\n\tcapChangeLst, err := capnp_patch.NewChange_List(seg, int32(len(p.Changes)))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif err := capPatch.SetChanges(capChangeLst); err != nil {\n\t\treturn err\n\t}\n\n\tfor idx, change := range p.Changes {\n\t\tcapCh, err := capnp_patch.NewChange(seg)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif err := change.toCapnpChange(seg, &capCh); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif err := capChangeLst.Set(idx, capCh); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// ToCapnp serializes a patch to capnproto message.\nfunc (p *Patch) ToCapnp() (*capnp.Message, error) {\n\tmsg, seg, err := capnp.NewMessage(capnp.SingleSegment(nil))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapPatch, err := capnp_patch.NewRootPatch(seg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn msg, p.toCapnpPatch(seg, capPatch)\n}\n\n// ToCapnp seriales patches to a capnproto message.\nfunc (ps Patches) ToCapnp() (*capnp.Message, error) {\n\tmsg, seg, err := capnp.NewMessage(capnp.SingleSegment(nil))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapPatches, err := capnp_patch.NewRootPatches(seg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapPatchLst, err := capnp_patch.NewPatch_List(seg, int32(len(ps)))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := capPatches.SetPatches(capPatchLst); err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor idx, p := range ps {\n\t\tcapPatch, err := capnp_patch.NewPatch(seg)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif err := p.toCapnpPatch(seg, capPatch); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif err := capPatchLst.Set(idx, capPatch); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn msg, nil\n}\n\nfunc (p *Patch) fromCapnpPatch(capPatch capnp_patch.Patch) error {\n\tp.FromIndex = capPatch.FromIndex()\n\tp.CurrIndex = capPatch.CurrIndex()\n\n\tcapChs, err := capPatch.Changes()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor idx := 0; idx < capChs.Len(); idx++ {\n\t\tch := &Change{}\n\t\tif err := ch.fromCapnpChange(capChs.At(idx)); err != nil {\n\t\t\treturn e.Wrapf(err, \"patch: from-capnp: change\")\n\t\t}\n\n\t\tp.Changes = append(p.Changes, ch)\n\t}\n\n\treturn nil\n}\n\n// FromCapnp deserializes `msg` into `p`.\nfunc (p *Patch) FromCapnp(msg *capnp.Message) error {\n\tcapPatch, err := capnp_patch.ReadRootPatch(msg)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn p.fromCapnpPatch(capPatch)\n}\n\n// FromCapnp deserializes `msg` into `ps`\nfunc (ps *Patches) FromCapnp(msg *capnp.Message) error {\n\tcapPatches, err := capnp_patch.ReadRootPatches(msg)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tcapPatchesLst, err := capPatches.Patches()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tnewPatches := Patches{}\n\tfor idx := 0; idx < capPatchesLst.Len(); idx++ {\n\t\tp := &Patch{}\n\t\tif err := p.fromCapnpPatch(capPatchesLst.At(idx)); err != nil {\n\t\t\treturn e.Wrapf(err, \"patches: from-capnp: patch\")\n\t\t}\n\n\t\tnewPatches = append(newPatches, p)\n\t}\n\n\t*ps = newPatches\n\treturn nil\n}\n\n// buildPrefixTrie builds a trie of prefixes that can be passed\nfunc buildPrefixTrie(prefixes []string) *trie.Node {\n\troot := trie.NewNode()\n\tfor _, prefix := range prefixes {\n\t\tif prefix == \"/\" {\n\t\t\troot.Data = true\n\t\t} else {\n\t\t\troot.Insert(prefix).Data = true\n\t\t}\n\t}\n\n\treturn root\n}\n\nfunc hasValidPrefix(root *trie.Node, path string) bool {\n\tif root.Data != nil && root.Data.(bool) == true {\n\t\treturn true\n\t}\n\n\tcurr := root\n\tfor _, elem := range trie.SplitPath(path) {\n\t\tcurr = curr.Lookup(elem)\n\n\t\t// No such children, not an allowed prefix.\n\t\tif curr == nil {\n\t\t\treturn false\n\t\t}\n\n\t\t// If it's a prefix node it's over.\n\t\tif curr.Data != nil && curr.Data.(bool) == true {\n\t\t\treturn true\n\t\t}\n\t}\n\n\treturn false\n}\n\nfunc filterInvalidMoveGhost(lkr *c.Linker, child n.Node, combCh *Change, prefixTrie *trie.Node) (bool, error) {\n\tif child.Type() != n.NodeTypeGhost || combCh.Mask&ChangeTypeMove == 0 {\n\t\treturn true, nil\n\t}\n\n\tmoveNd, _, err := lkr.MoveEntryPoint(child)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\tif moveNd == nil {\n\t\treturn false, nil\n\t}\n\n\tif !hasValidPrefix(prefixTrie, moveNd.Path()) {\n\t\t// The node was moved to the outside. Count it as removed.\n\t\tcombCh.Mask &= ^ChangeTypeMove\n\t\tcombCh.Mask |= ChangeTypeRemove\n\t\treturn true, nil\n\t}\n\n\treturn true, nil\n}\n\n// MakePatch creates a patch with all changes starting from `from`.\n// Patch will be created betweed `from` and `status` (current state)\n// It will only include nodes that are located under one of the prefixes in `prefixes`.\nfunc MakePatch(lkr *c.Linker, from *n.Commit, prefixes []string) (*Patch, error) {\n\tto, err := lkr.Status()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn MakePatchFromTo(lkr, from, to, prefixes)\n}\n\n// MakePatches is like MakePatch but produces several patches, not a compressed one.\nfunc MakePatches(lkr *c.Linker, from *n.Commit, prefixes []string) (Patches, error) {\n\tto, err := lkr.Status()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tpatches := []*Patch{}\n\n\tvar errSkip = errors.New(\"stop log\")\n\tvar prevCmt = to\n\n\t// TODO: Log API should offer something like errSkip itself.\n\terr = c.Log(lkr, to, func(cmt *n.Commit) error {\n\t\tif prevCmt.Index() == to.Index() {\n\t\t\t// First iteration.\n\t\t\tprevCmt = cmt\n\t\t\treturn nil\n\t\t}\n\n\t\tpatch, err := MakePatchFromTo(lkr, cmt, prevCmt, prefixes)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tpatches = append(patches, patch)\n\t\tprevCmt = cmt\n\n\t\tif cmt.Index() == from.Index() {\n\t\t\t// We've gone deep enough.\n\t\t\treturn errSkip\n\t\t}\n\n\t\treturn nil\n\t})\n\n\tif err != nil && err != errSkip {\n\t\treturn nil, err\n\t}\n\n\treturn patches, nil\n}\n\n// MakePatchFromTo makes a patch between two commits `from` (older one) and `to` (newer one)\nfunc MakePatchFromTo(lkr *c.Linker, from, to *n.Commit, prefixes []string) (*Patch, error) {\n\troot, err := to.Child(lkr, \"does not matter\") // child actually means Root for commits\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif from == nil {\n\t\treturn nil, e.New(\"The `from` commit is nil\")\n\t}\n\n\tif to == nil {\n\t\treturn nil, e.New(\"The `to` commit is nil\")\n\t}\n\n\tpatch := &Patch{\n\t\tFromIndex: from.Index(),\n\t\tCurrIndex: to.Index(),\n\t}\n\n\t// Shortcut: The patch CURR..CURR would be empty.\n\t// No need for further computations.\n\tif from.TreeHash().Equal(to.TreeHash()) {\n\t\treturn patch, nil\n\t}\n\n\t// Build a prefix trie to quickly check invalid paths.\n\t// This is not necessarily much faster, but runs in constant time.\n\tif prefixes == nil {\n\t\tprefixes = []string{\"/\"}\n\t}\n\tprefixTrie := buildPrefixTrie(prefixes)\n\n\terr = n.Walk(lkr, root, false, func(child n.Node) error {\n\t\tchildParentPath := path.Dir(child.Path())\n\t\tif len(prefixes) != 0 && !hasValidPrefix(prefixTrie, childParentPath) {\n\t\t\tlog.Debugf(\"Ignoring invalid prefix: %s\", childParentPath)\n\t\t\treturn nil\n\t\t}\n\n\t\t// Get all changes between `to` and `from`.\n\t\tchildModNode, ok := child.(n.ModNode)\n\t\tif !ok {\n\t\t\treturn e.Wrapf(ie.ErrBadNode, \"make-patch: walk\")\n\t\t}\n\n\t\tchanges, err := History(lkr, childModNode, to, from)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// No need to export empty history, abort early.\n\t\tif len(changes) == 0 {\n\t\t\treturn nil\n\t\t}\n\n\t\tcombCh := CombineChanges(changes)\n\n\t\t// Directories are a bit of a special case. We're only interested in them\n\t\t// when creating new, empty directories (n_children == 0) or if whole trees\n\t\t// were moved. In the latter case we need to also send a notice about that,\n\t\t// but we can leave out any other change.\n\t\tif child.Type() == n.NodeTypeDirectory {\n\t\t\tdir, ok := child.(*n.Directory)\n\t\t\tif !ok {\n\t\t\t\treturn e.Wrapf(ie.ErrBadNode, \"make-patch: dir\")\n\t\t\t}\n\n\t\t\tif combCh.Mask&ChangeTypeMove == 0 {\n\t\t\t\tif dir.NChildren() > 0 {\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tcombCh.Mask = ChangeTypeMove\n\t\t\t}\n\t\t}\n\n\t\t// Some special filtering needs to be done here. If it'a \"move\" ghost\n\t\t// we don't want to export it if the move goes outside our prefixes\n\t\t// (which would count as \"remove\"). or if we already reported a top\n\t\t// level directory that contains this move.\n\t\tisValid, err := filterInvalidMoveGhost(lkr, child, combCh, prefixTrie)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tlog.Debugf(\"combine: %v <= %v (valid %v)\", combCh, changes, isValid)\n\t\tif isValid && combCh.Mask != 0 {\n\t\t\tpatch.Changes = append(patch.Changes, combCh)\n\t\t}\n\n\t\treturn nil\n\t})\n\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Make sure the patch is applied in the right order.\n\t// The receiving site will sort it again, but it's better\n\t// to have it in the right order already.\n\tsort.Sort(patch)\n\n\tfor _, ch := range patch.Changes {\n\t\tlog.Debugf(\" change: %s\", ch)\n\t}\n\n\treturn patch, nil\n}\n\n// ApplyPatch applies the patch `p` to the linker `lkr`.\nfunc ApplyPatch(lkr *c.Linker, p *Patch) error {\n\tsort.Sort(p)\n\n\tfor _, change := range p.Changes {\n\t\tlog.Debugf(\"apply %s %v\", change, change.Curr.Type())\n\t\tif err := change.Replay(lkr); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n" }, { "path": "catfs/vcs/patch_test.go", "content": "package vcs\n\nimport (\n\t\"testing\"\n\n\tc \"github.com/sahib/brig/catfs/core\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestPatchMarshalling(t *testing.T) {\n\tc.WithDummyLinker(t, func(lkr *c.Linker) {\n\t\thead, err := lkr.Head()\n\t\trequire.Nil(t, err)\n\n\t\tcurr := c.MustTouch(t, lkr, \"/x\", 1)\n\t\tnext := c.MustCommit(t, lkr, \"hello\")\n\n\t\tchange1 := &Change{\n\t\t\tMask: ChangeTypeMove | ChangeTypeRemove,\n\t\t\tHead: head,\n\t\t\tNext: next,\n\t\t\tCurr: curr,\n\t\t\tMovedTo: \"/something1\",\n\t\t}\n\n\t\tc.MustModify(t, lkr, curr, 2)\n\t\tnextNext := c.MustCommit(t, lkr, \"hello\")\n\n\t\tchange2 := &Change{\n\t\t\tMask: ChangeTypeAdd | ChangeTypeModify,\n\t\t\tHead: next,\n\t\t\tNext: nextNext,\n\t\t\tCurr: curr,\n\t\t\tMovedTo: \"/something2\",\n\t\t}\n\n\t\tpatch := &Patch{\n\t\t\tFromIndex: head.Index(),\n\t\t\tChanges: []*Change{change2, change1},\n\t\t}\n\n\t\tmsg, err := patch.ToCapnp()\n\t\trequire.Nil(t, err)\n\n\t\tnewPatch := &Patch{}\n\t\trequire.Nil(t, newPatch.FromCapnp(msg))\n\n\t\trequire.Equal(t, patch, newPatch)\n\t})\n}\n\nfunc TestPrefixTrie(t *testing.T) {\n\tprefixes := []string{\n\t\t\"/a\",\n\t\t\"/b\",\n\t\t\"/c/d\",\n\t}\n\n\troot := buildPrefixTrie(prefixes)\n\trequire.True(t, hasValidPrefix(root, \"/a\"))\n\trequire.True(t, hasValidPrefix(root, \"/a/x/y/z\"))\n\trequire.True(t, hasValidPrefix(root, \"/b/c\"))\n\trequire.True(t, hasValidPrefix(root, \"/c/d/e\"))\n\n\trequire.False(t, hasValidPrefix(root, \"/c/e/d\"))\n\trequire.False(t, hasValidPrefix(root, \"/c/a/b\"))\n\trequire.False(t, hasValidPrefix(root, \"/\"))\n\trequire.False(t, hasValidPrefix(root, \"/d\"))\n}\n\nfunc TestMakePatch(t *testing.T) {\n\tc.WithLinkerPair(t, func(lkrSrc, lkrDst *c.Linker) {\n\t\tinit, err := lkrSrc.Head()\n\t\trequire.Nil(t, err)\n\n\t\tsrcX := c.MustTouch(t, lkrSrc, \"/x\", 1)\n\t\tsrcY := c.MustTouch(t, lkrSrc, \"/y\", 2)\n\t\tc.MustMkdir(t, lkrSrc, \"/sub\")\n\t\tc.MustMkdir(t, lkrSrc, \"/empty\")\n\t\tsrcZ := c.MustTouch(t, lkrSrc, \"/sub/z\", 3)\n\t\tc.MustCommit(t, lkrSrc, \"3 files\")\n\n\t\tpatch, err := MakePatch(lkrSrc, init, []string{\"/\"})\n\t\trequire.Nil(t, err)\n\n\t\trequire.Nil(t, ApplyPatch(lkrDst, patch))\n\t\tdstX, err := lkrDst.LookupFile(\"/x\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, dstX.ContentHash(), h.TestDummy(t, 1))\n\n\t\tdstY, err := lkrDst.LookupFile(\"/y\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, dstY.ContentHash(), h.TestDummy(t, 2))\n\n\t\tdstZ, err := lkrDst.LookupFile(\"/sub/z\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, dstZ.ContentHash(), h.TestDummy(t, 3))\n\n\t\t_, err = lkrDst.LookupDirectory(\"/empty\")\n\t\trequire.Nil(t, err)\n\n\t\t///////////////////\n\n\t\tc.MustModify(t, lkrSrc, srcX, 4)\n\t\tc.MustMove(t, lkrSrc, srcY, \"/y_moved\")\n\t\tc.MustRemove(t, lkrSrc, srcZ)\n\t\tc.MustTouch(t, lkrSrc, \"/empty/not_empty_anymore\", 42)\n\n\t\tpatch, err = MakePatch(lkrSrc, init, []string{\"/\"})\n\t\trequire.Nil(t, err)\n\t\trequire.Nil(t, ApplyPatch(lkrDst, patch))\n\n\t\tdstYMoved, err := lkrDst.LookupFile(\"/y_moved\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, dstYMoved.Path(), \"/y_moved\")\n\n\t\tdstYGhost, err := lkrDst.LookupGhost(\"/y\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, dstYGhost.Path(), \"/y\")\n\n\t\tdstZGhost, err := lkrDst.LookupGhost(\"/sub/z\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, dstZGhost.Path(), \"/sub/z\")\n\n\t\tdstNotEmptyFile, err := lkrDst.LookupFile(\"/empty/not_empty_anymore\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, dstNotEmptyFile.Path(), \"/empty/not_empty_anymore\")\n\t})\n}\n\nfunc TestMakePatchWithOrderConflict(t *testing.T) {\n\tc.WithLinkerPair(t, func(lkrSrc, lkrDst *c.Linker) {\n\t\tinit, err := lkrSrc.Head()\n\t\trequire.Nil(t, err)\n\n\t\tsrcX := c.MustTouch(t, lkrSrc, \"/x\", 1)\n\t\tsrcY := c.MustTouch(t, lkrSrc, \"/y\", 2)\n\t\tc.MustCommit(t, lkrSrc, \"pre-move\")\n\n\t\tc.MustMove(t, lkrSrc, srcX, \"/z\")\n\t\tc.MustMove(t, lkrSrc, srcY, \"/z\")\n\t\tc.MustCommit(t, lkrSrc, \"post-move\")\n\n\t\tpatch, err := MakePatch(lkrSrc, init, []string{\"/\"})\n\t\trequire.Nil(t, err)\n\n\t\t// All files should be mentioned in the patch.\n\t\t// x and y are ghosts, z is the only real file.\n\t\t// Since y was moved last it has a move marker, x not.\n\t\trequire.Equal(t, \"/x\", patch.Changes[1].Curr.Path())\n\t\trequire.Equal(t, \"\", patch.Changes[1].MovedTo)\n\t\trequire.Equal(t, \"\", patch.Changes[1].WasPreviouslyAt)\n\t\trequire.Equal(t, n.NodeTypeGhost, patch.Changes[1].Curr.Type())\n\n\t\trequire.Equal(t, \"/y\", patch.Changes[0].Curr.Path())\n\t\trequire.Equal(t, \"/z\", patch.Changes[0].MovedTo)\n\t\trequire.Equal(t, n.NodeTypeGhost, patch.Changes[0].Curr.Type())\n\n\t\trequire.Equal(t, \"/z\", patch.Changes[2].Curr.Path())\n\t\trequire.Equal(t, \"\", patch.Changes[2].MovedTo)\n\t\trequire.Equal(t, \"/y\", patch.Changes[2].WasPreviouslyAt)\n\t\trequire.Equal(t, n.NodeTypeFile, patch.Changes[2].Curr.Type())\n\n\t\trequire.Nil(t, ApplyPatch(lkrDst, patch))\n\t\tdstZ, err := lkrDst.LookupFile(\"/z\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, dstZ.ContentHash(), h.TestDummy(t, 2))\n\n\t\tdstX, err := lkrDst.LookupGhost(\"/x\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, n.NodeTypeGhost, dstX.Type())\n\n\t\tdstY, err := lkrDst.LookupGhost(\"/y\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, n.NodeTypeGhost, dstY.Type())\n\t})\n}\n\n// Move all children of a directory to another location.\nfunc TestMakePatchDirMoveAllChildren(t *testing.T) {\n\tc.WithLinkerPair(t, func(lkrSrc, lkrDst *c.Linker) {\n\t\tinit, err := lkrSrc.Head()\n\t\trequire.Nil(t, err)\n\n\t\tc.MustMkdir(t, lkrSrc, \"/src\")\n\t\tsubX := c.MustTouch(t, lkrSrc, \"/src/x\", 1)\n\t\tsubY := c.MustTouch(t, lkrSrc, \"/src/y\", 2)\n\t\tpreMove := c.MustCommit(t, lkrSrc, \"base\")\n\n\t\tpatch1, err := MakePatch(lkrSrc, init, []string{\"/\"})\n\t\trequire.Nil(t, err)\n\n\t\trequire.Nil(t, ApplyPatch(lkrDst, patch1))\n\t\tsrcDir, err := lkrDst.LookupDirectory(\"/src\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, 2, srcDir.NChildren())\n\n\t\t///////////\n\n\t\tc.MustMkdir(t, lkrSrc, \"/dst\")\n\t\tc.MustMove(t, lkrSrc, subX, \"/dst/x\")\n\t\tc.MustMove(t, lkrSrc, subY, \"/dst/y\")\n\t\tc.MustCommit(t, lkrSrc, \"post-move\")\n\n\t\tpatch2, err := MakePatch(lkrSrc, preMove, []string{\"/\"})\n\t\trequire.Nil(t, err)\n\n\t\trequire.Nil(t, ApplyPatch(lkrDst, patch2))\n\n\t\tsrcDir, err = lkrDst.LookupDirectory(\"/src\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, 2, srcDir.NChildren())\n\n\t\t_, err = lkrDst.LookupGhost(\"/src/x\")\n\t\trequire.Nil(t, err)\n\t\t_, err = lkrDst.LookupGhost(\"/src/x\")\n\t\trequire.Nil(t, err)\n\n\t\t_, err = lkrDst.LookupFile(\"/dst/x\")\n\t\trequire.Nil(t, err)\n\t\t_, err = lkrDst.LookupFile(\"/dst/x\")\n\t\trequire.Nil(t, err)\n\t})\n}\n\n// Move a directory completely, not just its contents.\nfunc TestMakePatchDirMoveCompletely(t *testing.T) {\n\tc.WithLinkerPair(t, func(lkrSrc, lkrDst *c.Linker) {\n\t\tinit, err := lkrSrc.Head()\n\t\trequire.Nil(t, err)\n\n\t\trealSrcDir := c.MustMkdir(t, lkrSrc, \"/src\")\n\t\tc.MustTouch(t, lkrSrc, \"/src/x\", 1)\n\t\tc.MustTouch(t, lkrSrc, \"/src/y\", 2)\n\t\tpreMove := c.MustCommit(t, lkrSrc, \"base\")\n\n\t\tpatch1, err := MakePatch(lkrSrc, init, []string{\"/\"})\n\t\trequire.Nil(t, err)\n\n\t\trequire.Nil(t, ApplyPatch(lkrDst, patch1))\n\t\tsrcDir, err := lkrDst.LookupDirectory(\"/src\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, 2, srcDir.NChildren())\n\n\t\t///////////\n\n\t\tc.MustMove(t, lkrSrc, realSrcDir, \"/dst\")\n\t\tc.MustCommit(t, lkrSrc, \"post-move\")\n\n\t\tpatch2, err := MakePatch(lkrSrc, preMove, []string{\"/\"})\n\t\trequire.Nil(t, err)\n\n\t\trequire.Nil(t, ApplyPatch(lkrDst, patch2))\n\n\t\tsrcDirGhost, err := lkrDst.LookupGhost(\"/src\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, 2, srcDirGhost.NChildren())\n\n\t\t_, err = lkrDst.LookupNode(\"/src/x\")\n\t\trequire.NotNil(t, err)\n\t\t_, err = lkrDst.LookupNode(\"/src/x\")\n\t\trequire.NotNil(t, err)\n\n\t\t_, err = lkrDst.LookupFile(\"/dst/x\")\n\t\trequire.Nil(t, err)\n\t\t_, err = lkrDst.LookupFile(\"/dst/x\")\n\t\trequire.Nil(t, err)\n\t})\n}\n\nfunc TestSyncPartialTwiceWithMovedFile(t *testing.T) {\n\tc.WithLinkerPair(t, func(lkrAli, lkrBob *c.Linker) {\n\t\taliNd, _ := c.MustTouchAndCommit(t, lkrAli, \"/ali-file\", 1)\n\t\tc.MustTouchAndCommit(t, lkrBob, \"/bob-file\", 2)\n\n\t\trequire.Nil(t, Sync(lkrAli, lkrBob, nil))\n\t\trequire.Nil(t, Sync(lkrBob, lkrAli, nil))\n\n\t\tbeforeMove, err := lkrAli.Head()\n\t\trequire.Nil(t, err)\n\t\tc.MustMove(t, lkrAli, aliNd, \"/bali-bile\")\n\t\tc.MustCommit(t, lkrAli, \"after move\")\n\n\t\t// do commit before.\n\t\tpatch, err := MakePatch(lkrAli, beforeMove, nil)\n\t\trequire.Nil(t, err)\n\n\t\t// Apply the patch on bob's side.\n\t\trequire.Nil(t, ApplyPatch(lkrBob, patch))\n\t\tc.MustCommit(t, lkrBob, \"after patch\")\n\n\t\tdiff, err := MakeDiff(lkrBob, lkrAli, nil, nil, nil)\n\t\trequire.Nil(t, err)\n\n\t\trequire.Len(t, diff.Added, 0)\n\t\trequire.Len(t, diff.Removed, 0)\n\t\trequire.Len(t, diff.Moved, 0)\n\t\trequire.Len(t, diff.Conflict, 0)\n\t\trequire.Len(t, diff.Merged, 0)\n\t\trequire.Len(t, diff.Ignored, 0)\n\t})\n}\n" }, { "path": "catfs/vcs/reset.go", "content": "package vcs\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"path\"\n\n\te \"github.com/pkg/errors\"\n\tc \"github.com/sahib/brig/catfs/core\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n)\n\nfunc findPathAt(lkr *c.Linker, cmt *n.Commit, path string) (string, error) {\n\tnd, err := lkr.LookupModNode(path)\n\tif err != nil && !ie.IsNoSuchFileError(err) {\n\t\treturn \"\", err\n\t}\n\n\tif ie.IsNoSuchFileError(err) {\n\t\t// The file does not exist in the current commit,\n\t\t// so user probably knows that it had this path before.\n\t\treturn path, nil\n\t}\n\n\tstatus, err := lkr.Status()\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\twalker := NewHistoryWalker(lkr, status, nd)\n\tfor walker.Next() {\n\t\tstate := walker.State()\n\t\tif state.Head.TreeHash().Equal(cmt.TreeHash()) {\n\t\t\treturn state.Curr.Path(), nil\n\t\t}\n\t}\n\n\tif err := walker.Err(); err != nil {\n\t\treturn \"\", err\n\t}\n\n\t// Take the current path as best guess.\n\treturn path, nil\n}\n\nfunc clearPath(lkr *c.Linker, ndPath string) (*n.Directory, error) {\n\tnd, err := lkr.LookupModNode(ndPath)\n\tisNoSuchFile := ie.IsNoSuchFileError(err)\n\n\tif err != nil && !isNoSuchFile {\n\t\treturn nil, err\n\t}\n\n\tvar par *n.Directory\n\tif ndPath != \"/\" {\n\t\tpar, err = lkr.LookupDirectory(path.Dir(ndPath))\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tif par == nil {\n\t\treturn nil, fmt.Errorf(\n\t\t\t\"checkout by commit if you want to checkout previous roots\",\n\t\t)\n\t}\n\n\t// The node does currently not exist (and the user wants to bring it back)\n\tif isNoSuchFile {\n\t\treturn par, nil\n\t}\n\n\terr = n.Walk(lkr, nd, true, func(child n.Node) error {\n\t\tlkr.MemIndexPurge(child)\n\t\treturn nil\n\t})\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := par.RemoveChild(lkr, nd); err != nil {\n\t\treturn nil, err\n\t}\n\n\tlkr.MemIndexPurge(nd)\n\treturn par, lkr.StageNode(par)\n}\n\n// ResetNode resets a certain file to the state it had in cmt. If the file\n// did not exist back then, it will be deleted. `nd` is usually retrieved by\n// calling ResolveNode() and sorts.\n//\n// A special case occurs when the file was moved we reset to.\n// In this case the state of the old node (at the old path)\n// is being written to the node at the new path.\n// This is the more obvious choice to the user when he types:\n//\n// $ brig reset HEAD^ i-was-somewhere-else-before # name does not change.\n//\n// This method returns the old node (or nil if none) and any possible error.\nfunc ResetNode(lkr *c.Linker, cmt *n.Commit, currPath string) (n.Node, error) {\n\troot, err := lkr.DirectoryByHash(cmt.Root())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif root == nil {\n\t\treturn nil, errors.New(\"no root to reset to\")\n\t}\n\n\t// Find out the old path of `currPath` at `cmt`.\n\t// It might have changed due to moves.\n\toldPath, err := findPathAt(lkr, cmt, currPath)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\toldNode, err := root.Lookup(lkr, oldPath)\n\tif err != nil && !ie.IsNoSuchFileError(err) {\n\t\treturn nil, err\n\t}\n\n\t// Make sure that all write related action happen in one go:\n\treturn oldNode, lkr.Atomic(func() (bool, error) {\n\t\t// Remove the node that is present at the current path:\n\t\tpar, err := clearPath(lkr, currPath)\n\t\tif err != nil {\n\t\t\treturn true, err\n\t\t}\n\n\t\t// old Node might not have yet existed back then.\n\t\t// If so, simply do not re-add it.\n\t\tif oldNode != nil {\n\t\t\toldModNode, ok := oldNode.(n.ModNode)\n\t\t\tif !ok {\n\t\t\t\treturn true, e.Wrapf(ie.ErrBadNode, \"reset file\")\n\t\t\t}\n\n\t\t\t// If the old node was at a different location,\n\t\t\t// we need to modify its path.\n\t\t\toldModNode.SetName(path.Base(currPath))\n\t\t\tif err := oldModNode.SetParent(lkr, par); err != nil {\n\t\t\t\treturn true, err\n\t\t\t}\n\n\t\t\tif err := oldModNode.NotifyMove(lkr, par, oldModNode.Path()); err != nil {\n\t\t\t\treturn true, err\n\t\t\t}\n\n\t\t\tif err := lkr.StageNode(oldNode); err != nil {\n\t\t\t\treturn true, err\n\t\t\t}\n\t\t}\n\n\t\treturn false, nil\n\t})\n}\n" }, { "path": "catfs/vcs/reset_test.go", "content": "package vcs\n\nimport (\n\t\"testing\"\n\n\tc \"github.com/sahib/brig/catfs/core\"\n\t\"github.com/sahib/brig/catfs/db\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestResetFile(t *testing.T) {\n\tc.WithDummyKv(t, func(kv db.Database) {\n\t\tlkr := c.NewLinker(kv)\n\t\tif err := lkr.MakeCommit(n.AuthorOfStage, \"initial commit\"); err != nil {\n\t\t\tt.Fatalf(\"Initial commit failed: %v\", err)\n\t\t}\n\n\t\tinitCmt, err := lkr.Head()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to get initial head\")\n\t\t}\n\n\t\troot, err := lkr.Root()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Getting root failed: %v\", err)\n\t\t}\n\n\t\tfile := n.NewEmptyFile(root, \"cat.png\", \"u\", 3)\n\n\t\tc.MustModify(t, lkr, file, 1)\n\t\toldFileHash := file.TreeHash().Clone()\n\n\t\tif err := lkr.MakeCommit(n.AuthorOfStage, \"second commit\"); err != nil {\n\t\t\tt.Fatalf(\"Failed to make second commit: %v\", err)\n\t\t}\n\n\t\tc.MustModify(t, lkr, file, 2)\n\t\theadFileHash := file.TreeHash().Clone()\n\n\t\tif err := lkr.MakeCommit(n.AuthorOfStage, \"third commit\"); err != nil {\n\t\t\tt.Fatalf(\"Failed to make third commit: %v\", err)\n\t\t}\n\n\t\thead, err := lkr.Head()\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to get HEAD: %v\", err)\n\t\t}\n\n\t\tlastCommitNd, err := head.Parent(lkr)\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to get second commit: %v\", err)\n\t\t}\n\n\t\tlastCommit := lastCommitNd.(*n.Commit)\n\n\t\tif _, err := ResetNode(lkr, lastCommit, \"/cat.png\"); err != nil {\n\t\t\tt.Fatalf(\"Failed to checkout file before commit: %v\", err)\n\t\t}\n\n\t\tlastVersion, err := lkr.LookupFile(\"/cat.png\")\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to lookup /cat.png post checkout\")\n\t\t}\n\n\t\tif !lastVersion.TreeHash().Equal(oldFileHash) {\n\t\t\tt.Fatalf(\"Hash of checkout'd file is not from second commit\")\n\t\t}\n\n\t\tif lastVersion.Size() != 1 {\n\t\t\tt.Fatalf(\"Size of checkout'd file is not from second commit\")\n\t\t}\n\n\t\tif _, err := ResetNode(lkr, initCmt, \"/cat.png\"); err != nil {\n\t\t\tt.Fatalf(\"Failed to checkout file at init: %v\", err)\n\t\t}\n\n\t\t_, err = lkr.LookupFile(\"/cat.png\")\n\t\tif !ie.IsNoSuchFileError(err) {\n\t\t\tt.Fatalf(\"Different error: %v\", err)\n\t\t}\n\n\t\tif _, err := ResetNode(lkr, head, \"/cat.png\"); err != nil {\n\t\t\tt.Fatalf(\"Failed to checkout file at head: %v\", err)\n\t\t}\n\n\t\theadVersion, err := lkr.LookupFile(\"/cat.png\")\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to lookup /cat.png post checkout\")\n\t\t}\n\n\t\tif !headVersion.TreeHash().Equal(headFileHash) {\n\t\t\tt.Fatalf(\n\t\t\t\t\"Hash differs between new and head reset: %v != %v\",\n\t\t\t\theadVersion.TreeHash(),\n\t\t\t\theadFileHash,\n\t\t\t)\n\t\t}\n\t})\n}\n\nfunc TestFindPathAt(t *testing.T) {\n\tc.WithDummyLinker(t, func(lkr *c.Linker) {\n\t\tnd := c.MustTouch(t, lkr, \"/x\", 1)\n\t\tc1 := c.MustCommit(t, lkr, \"1\")\n\t\tc.MustMove(t, lkr, nd, \"/y\")\n\t\tc.MustCommit(t, lkr, \"2\")\n\n\t\toldPath, err := findPathAt(lkr, c1, \"/y\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, \"/x\", oldPath)\n\t})\n}\n\n// Reset a file that was moved in earlier incarnations.\nfunc TestResetMovedFile(t *testing.T) {\n\tc.WithDummyLinker(t, func(lkr *c.Linker) {\n\t\tsub := c.MustMkdir(t, lkr, \"/sub\")\n\t\tnd := c.MustTouch(t, lkr, \"/sub/x\", 1)\n\t\tc1 := c.MustCommit(t, lkr, \"1\")\n\t\tc.MustMove(t, lkr, nd, \"/y\")\n\t\tc.MustModify(t, lkr, nd, 2)\n\t\tc.MustCommit(t, lkr, \"2\")\n\n\t\t// This should reset /y to content=1.\n\t\t_, err := ResetNode(lkr, c1, \"/y\")\n\t\trequire.Nil(t, err)\n\n\t\troot, err := lkr.Root()\n\t\trequire.Nil(t, err)\n\n\t\tchildren, err := root.ChildrenSorted(lkr)\n\t\trequire.Nil(t, err)\n\t\trequire.Len(t, children, 2)\n\t\trequire.Equal(t, children[0].Type(), n.NodeType(n.NodeTypeDirectory))\n\t\trequire.Equal(t, children[0].Path(), \"/sub\")\n\n\t\trequire.Equal(t, children[1].Type(), n.NodeType(n.NodeTypeFile))\n\t\trequire.Equal(t, children[1].Path(), \"/y\")\n\t\trequire.Equal(t, children[1].BackendHash(), h.TestDummy(t, 1))\n\n\t\tsubChildren, err := sub.ChildrenSorted(lkr)\n\t\trequire.Nil(t, err)\n\t\trequire.Len(t, subChildren, 1)\n\t\trequire.Equal(t, subChildren[0].Type(), n.NodeType(n.NodeTypeGhost))\n\t\trequire.Equal(t, subChildren[0].Path(), \"/sub/x\")\n\t})\n}\n" }, { "path": "catfs/vcs/resolve.go", "content": "package vcs\n\n// This package implements brig's sync algorithm which I called, in a burst of\n// modesty, \"bright\". (Not because it's or I'm very bright, but because it\n// starts with brig...)\n//\n// The sync algorithm tries to handle the following special cases:\n// - Propagate moves (most of them, at least)\n// - Propagate deletes (configurable?)\n// - Also sync empty directories.\n//\n// Terminology:\n// - Destination (short \"dst\") is used to reference our own storage.\n// - Source (short: \"src\") is used to reference the remote storage.\n//\n// The sync algorithm can be roughly divided in 4 stages:\n// - Stage 1: \"Move Marking\":\n// Iterate over all ghosts in the tree and check if they were either moved\n// (has sibling) or removed (has no sibling). In case of directories, the\n// second mapping stage is already executed.\n//\n// - Stage 2: \"Mapping\":\n// Finding pairs of files that possibly adding, merging or conflict handling.\n// Equal files will already be sorted out at this point. Every already\n// visited node in the remote linker will be marked. The mapping algorithm\n// starts at the root node and uses the attributes of the merkle trees\n// (same hash = same content) to skip over same parts.\n//\n// - Stage 3: \"Resolving\":\n// For each file a decision needs to be made. This decision defines the next step\n// and can be one of the following.\n//\n// - The file was added on the remote, we should add it to -> Add them.\n// - The file was removed on the remote, we might want to also delete it.\n// - The file was only moved on the remote node, we might want to moev it also.\n// - The file has compatible changes on the both sides. -> Merge them.\n// - The file was incompatible changes on both sides -> Do conflict resolution.\n// - The nodes have differing types (directory vs files). Report them.\n//\n// - Stage 4: \"Handling\"\n// Only at this stage \"sync\" and \"diff\" differ.\n// Sync will take the the files from Stage 3 and add/remove/merge files.\n// Diff will create a report out of those files and also includes files that\n// are simply missing on the source side (but do not need to be removed).\n//\n// Everything except Stage 4 is read-only. If a user wants to only show the diff\n// between two linkers, he just prints what would be done instead of actually doing it.\n// This makes the diff and sync implementation share most of it's code.\n\nimport (\n\t\"fmt\"\n\t\"path\"\n\t\"regexp\"\n\t\"strings\"\n\n\te \"github.com/pkg/errors\"\n\tc \"github.com/sahib/brig/catfs/core\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\nvar (\n\tconflictNodePattern = regexp.MustCompile(`/.*\\.conflict\\.\\d+`)\n)\n\n// executor is the interface that executes the actual action\n// needed to perform the sync (see \"phase 4\" in the package doc)\ntype executor interface {\n\thandleAdd(src n.ModNode) error\n\thandleRemove(dst n.ModNode) error\n\thandleMissing(dst n.ModNode) error\n\thandleMove(src, dst n.ModNode) error\n\thandleConflictNode(src n.ModNode) error\n\thandleTypeConflict(src, dst n.ModNode) error\n\thandleMerge(src, dst n.ModNode, srcMask, dstMask ChangeType) error\n\thandleConflict(src, dst n.ModNode, srcMask, dstMask ChangeType) error\n}\n\n//////////////////////////////////////////////\n// IMPLEMENTATION OF ACTUAL DECISION MAKING //\n//////////////////////////////////////////////\n\ntype resolver struct {\n\tlkrSrc *c.Linker\n\tlkrDst *c.Linker\n\n\t// What points should be resolved\n\tdstHead *n.Commit\n\tsrcHead *n.Commit\n\n\t// cached attributes:\n\tdstMergeCmt *n.Commit\n\tsrcMergeCmt *n.Commit\n\n\t// actual executor based on the decision\n\texec executor\n}\n\nfunc newResolver(lkrSrc, lkrDst *c.Linker, srcHead, dstHead *n.Commit, exec executor) (*resolver, error) {\n\tvar err error\n\tif srcHead == nil {\n\t\tsrcHead, err = lkrSrc.Status()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tif dstHead == nil {\n\t\tdstHead, err = lkrDst.Status()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn &resolver{\n\t\tlkrSrc: lkrSrc,\n\t\tlkrDst: lkrDst,\n\t\tsrcHead: srcHead,\n\t\tdstHead: dstHead,\n\t\texec: exec,\n\t}, nil\n}\n\nfunc (rv *resolver) resolve() error {\n\tsrcRoot, err := rv.lkrSrc.DirectoryByHash(rv.srcHead.Root())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif err := rv.cacheLastCommonMerge(); err != nil {\n\t\treturn e.Wrapf(err, \"failed to find last common merge\")\n\t}\n\n\tmapper, err := NewMapper(rv.lkrSrc, rv.lkrDst, rv.srcHead, rv.dstHead, srcRoot)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tmappings := []MapPair{}\n\terr = mapper.Map(func(pair MapPair) error {\n\t\tmappings = append(mappings, pair)\n\t\treturn nil\n\t})\n\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor _, pair := range mappings {\n\t\tif err := rv.decide(pair); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (rv *resolver) cacheLastCommonMerge() error {\n\tsrcOwner, err := rv.lkrSrc.Owner()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tcurrHead := rv.dstHead\n\n\tfor currHead != nil {\n\t\twith, srcRef := currHead.MergeMarker()\n\t\tif with == srcOwner {\n\t\t\tsrcHead, err := rv.lkrSrc.CommitByHash(srcRef)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tdebugf(\"last merge found: %v = %s\", with, srcRef)\n\t\t\trv.dstMergeCmt = currHead\n\t\t\trv.srcMergeCmt = srcHead\n\t\t}\n\n\t\tprevHeadNode, err := currHead.Parent(rv.lkrDst)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif prevHeadNode == nil {\n\t\t\tbreak\n\t\t}\n\n\t\tnewDstHead, ok := prevHeadNode.(*n.Commit)\n\t\tif !ok {\n\t\t\treturn ie.ErrBadNode\n\t\t}\n\n\t\tcurrHead = newDstHead\n\t}\n\n\treturn nil\n}\n\n// isConflictPath will return true if the file or directory was created\n// as conflict file in case of a merge conflicts.\nfunc isConflictPath(path string) bool {\n\treturn conflictNodePattern.MatchString(path)\n}\n\n// hasConflictFile reports if we already created a conflict file for `dstNd`.\nfunc (rv *resolver) hasConflictFile(dstNd n.ModNode) (bool, error) {\n\tparent, err := rv.lkrDst.LookupDirectory(path.Dir(dstNd.Path()))\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\t// Assumption: The original node and its conflict fil\n\t// will be always on the same level. If this change,\n\t// the logic here has to change also.\n\tchildren, err := parent.ChildrenSorted(rv.lkrDst)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\tfor _, child := range children {\n\t\tif child.Type() == n.NodeTypeGhost {\n\t\t\tcontinue\n\t\t}\n\t\tif isConflictPath(child.Path()) {\n\t\t\t// Also check if the conflict file belongs to our node:\n\t\t\treturn strings.HasPrefix(child.Path(), dstNd.Path()), nil\n\t\t}\n\t}\n\n\t// None found, assume we do not have a conflict file (yet)\n\treturn false, nil\n}\n\n// hasConflicts is always called when two nodes are on both sides and they do\n// not have the same hash. In the best case, both have compatible changes and\n// can be merged, otherwise a user defined conflict strategy has to be applied.\nfunc (rv *resolver) hasConflicts(src, dst n.ModNode) (bool, ChangeType, ChangeType, error) {\n\t// Nodes with same hashes are no conflicts...\n\t// (tree hash is also influenced by content)\n\tif src.TreeHash().Equal(dst.TreeHash()) {\n\t\treturn false, 0, 0, nil\n\t}\n\n\tsrcHist, err := History(rv.lkrSrc, src, rv.srcHead, rv.srcMergeCmt)\n\tif err != nil {\n\t\treturn false, 0, 0, e.Wrapf(err, \"history src\")\n\t}\n\n\tdstHist, err := History(rv.lkrDst, dst, rv.dstHead, rv.dstMergeCmt)\n\tif err != nil {\n\t\treturn false, 0, 0, e.Wrapf(err, \"history dst\")\n\t}\n\n\t// This loop can be optimized if the need arises:\n\tcommonRootFound := false\n\tsrcRoot, dstRoot := len(srcHist), len(dstHist)\n\n\tfor srcIdx := 0; srcIdx < len(srcHist) && !commonRootFound; srcIdx++ {\n\t\tfor dstIdx := 0; dstIdx < len(dstHist) && !commonRootFound; dstIdx++ {\n\t\t\tsrcChange, dstChange := srcHist[srcIdx], dstHist[dstIdx]\n\n\t\t\tif srcChange.Curr.ContentHash().Equal(dstChange.Curr.ContentHash()) {\n\t\t\t\tsrcRoot, dstRoot = srcIdx, dstIdx\n\t\t\t\tcommonRootFound = true\n\t\t\t}\n\t\t}\n\t}\n\n\tsrcHist = srcHist[:srcRoot]\n\tdstHist = dstHist[:dstRoot]\n\n\t// Compute the combination of all changes:\n\tvar srcMask, dstMask ChangeType\n\tfor _, change := range srcHist {\n\t\tsrcMask |= change.Mask\n\t}\n\tfor _, change := range dstHist {\n\t\tdstMask |= change.Mask\n\t}\n\n\tif len(srcHist) == 0 && len(dstHist) == 0 {\n\t\treturn false, 0, 0, nil\n\t}\n\n\t// Handle a few lucky cases:\n\tif len(srcHist) > 0 && len(dstHist) == 0 {\n\t\t// We can \"fast forward\" our node.\n\t\t// There are only remote changes for this file.\n\t\treturn false, srcMask, dstMask, nil\n\n\t}\n\tif len(srcHist) == 0 && len(dstHist) > 0 {\n\t\t// Only our side has changes. We can consider this node as merged.\n\t\treturn false, 0, 0, nil\n\t}\n\n\t// Both sides have changes. Now we need to figure out if they are compatible.\n\t// We do this simply by OR-ing all changes on both side to an individual mask\n\t// and check if those can be applied on top of dst's current state.\n\tif !dstMask.IsCompatible(srcMask) {\n\t\t// The changes are not compatible.\n\t\t// We need to apply a conflict resolution strategy.\n\t\treturn true, srcMask, dstMask, nil\n\t}\n\n\t// No conflict. We can merge src and dst.\n\treturn false, srcMask, dstMask, nil\n}\n\nfunc pathOrNil(nd n.Node) string {\n\tif nd == nil {\n\t\treturn \"nil\"\n\t}\n\n\treturn nd.Path()\n}\n\nfunc (rv *resolver) decide(pair MapPair) error {\n\tlog.Debugf(\n\t\t\"Deciding pair: src=%v dst=%v\",\n\t\tpathOrNil(pair.Src),\n\t\tpathOrNil(pair.Dst),\n\t)\n\n\tif pair.Src == nil && pair.Dst == nil {\n\t\treturn fmt.Errorf(\"Received completely empty mapping; ignoring\")\n\t}\n\n\tif pair.Src != nil && isConflictPath(pair.Src.Path()) {\n\t\treturn rv.exec.handleConflictNode(pair.Src)\n\t}\n\n\tif pair.Dst != nil && isConflictPath(pair.Dst.Path()) {\n\t\treturn rv.exec.handleConflictNode(pair.Dst)\n\t}\n\n\tif pair.SrcWasMoved {\n\t\treturn rv.exec.handleMove(pair.Src, pair.Dst)\n\t}\n\n\tif pair.Src == nil {\n\t\tif pair.SrcWasRemoved {\n\t\t\treturn rv.exec.handleRemove(pair.Dst)\n\t\t}\n\n\t\treturn rv.exec.handleMissing(pair.Dst)\n\t}\n\n\tif pair.Dst == nil {\n\t\treturn rv.exec.handleAdd(pair.Src)\n\t}\n\n\tif pair.TypeMismatch {\n\t\tdebugf(\n\t\t\t\"%s is a %s and %s a %s; ignoring\",\n\t\t\tpair.Src.Path(), pair.Src.Type(),\n\t\t\tpair.Dst.Path(), pair.Dst.Type(),\n\t\t)\n\t\treturn rv.exec.handleTypeConflict(pair.Src, pair.Dst)\n\t}\n\n\thasConflicts, srcMask, dstMask, err := rv.hasConflicts(pair.Src, pair.Dst)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif hasConflicts {\n\t\treturn rv.exec.handleConflict(pair.Src, pair.Dst, srcMask, dstMask)\n\t}\n\n\thasConflictFile, err := rv.hasConflictFile(pair.Dst)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif hasConflictFile {\n\t\treturn nil\n\t}\n\n\t// handleMerge needs the masks to decide what path / content to choose.\n\treturn rv.exec.handleMerge(pair.Src, pair.Dst, srcMask, dstMask)\n}\n" }, { "path": "catfs/vcs/resolve_test.go", "content": "package vcs\n\nimport (\n\t\"testing\"\n\n\tc \"github.com/sahib/brig/catfs/core\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\t\"github.com/stretchr/testify/require\"\n)\n\ntype expect struct {\n\tdstMergeCmt *n.Commit\n\tsrcMergeCmt *n.Commit\n\n\tsrcFile *n.File\n\tdstFile *n.File\n\n\terr error\n\tresult bool\n}\n\nfunc setupResolveBasicNoConflict(t *testing.T, lkrSrc, lkrDst *c.Linker) *expect {\n\tsrc, _ := c.MustTouchAndCommit(t, lkrSrc, \"/x.png\", 1)\n\tdst, _ := c.MustTouchAndCommit(t, lkrDst, \"/x.png\", 2)\n\n\treturn &expect{\n\t\tdstMergeCmt: nil,\n\t\tsrcMergeCmt: nil,\n\t\tsrcFile: src,\n\t\tdstFile: dst,\n\t\terr: nil,\n\t\tresult: false,\n\t}\n}\n\nfunc TestHasConflicts(t *testing.T) {\n\tt.Parallel()\n\n\ttcs := []struct {\n\t\tname string\n\t\tsetup func(t *testing.T, lkrSrc, lkrDst *c.Linker) *expect\n\t}{\n\t\t{\n\t\t\tname: \"basic-no-conflict-file\",\n\t\t\tsetup: setupResolveBasicNoConflict,\n\t\t},\n\t}\n\n\tfor _, tc := range tcs {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tt.Parallel()\n\n\t\t\tc.WithLinkerPair(t, func(lkrSrc, lkrDst *c.Linker) {\n\t\t\t\texpect := tc.setup(t, lkrSrc, lkrDst)\n\n\t\t\t\tsyncer, err := newResolver(lkrSrc, lkrDst, nil, nil, nil)\n\t\t\t\trequire.Nil(t, err)\n\n\t\t\t\tif err := syncer.cacheLastCommonMerge(); err != nil {\n\t\t\t\t\tt.Fatalf(\"Failed to find last common merge.\")\n\t\t\t\t}\n\n\t\t\t\trequire.Equal(\n\t\t\t\t\tt,\n\t\t\t\t\texpect.dstMergeCmt,\n\t\t\t\t\tsyncer.dstMergeCmt,\n\t\t\t\t\t\"dst merge marker\",\n\t\t\t\t)\n\t\t\t\trequire.Equal(\n\t\t\t\t\tt,\n\t\t\t\t\texpect.srcMergeCmt,\n\t\t\t\t\tsyncer.srcMergeCmt,\n\t\t\t\t\t\"src merge marker\",\n\t\t\t\t)\n\n\t\t\t\tresult, _, _, err := syncer.hasConflicts(\n\t\t\t\t\texpect.srcFile,\n\t\t\t\t\texpect.dstFile,\n\t\t\t\t)\n\t\t\t\tif expect.err != err {\n\t\t\t\t\tt.Fatalf(\n\t\t\t\t\t\t\"Resolve failed with wrong error: %v (want %v)\",\n\t\t\t\t\t\terr, expect.err)\n\t\t\t\t}\n\n\t\t\t\tif expect.result == result {\n\t\t\t\t\tt.Fatalf(\n\t\t\t\t\t\t\"resolve did not deliver the expected. Want %v, but got %v\",\n\t\t\t\t\t\texpect.result,\n\t\t\t\t\t\tresult,\n\t\t\t\t\t)\n\t\t\t\t}\n\t\t\t})\n\t\t})\n\t}\n}\n" }, { "path": "catfs/vcs/sync.go", "content": "package vcs\n\nimport (\n\t\"fmt\"\n\t\"path\"\n\n\te \"github.com/pkg/errors\"\n\tc \"github.com/sahib/brig/catfs/core\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\nconst (\n\t// ConflictStragetyMarker creates marker files for each conflict.\n\tConflictStragetyMarker = iota\n\n\t// ConflictStragetyIgnore ignores conflicts totally.\n\tConflictStragetyIgnore\n\n\t// ConflictStragetyEmbrace takes the version of the remote.\n\tConflictStragetyEmbrace\n\n\t// ConflictStragetyUnknown should be used when the strategy is not clear.\n\tConflictStragetyUnknown\n)\n\n// ConflictStrategy defines what conflict strategy to apply in case of\n// nodes with different content hashes.\ntype ConflictStrategy int\n\nfunc (cs ConflictStrategy) String() string {\n\tswitch cs {\n\tcase ConflictStragetyMarker:\n\t\treturn \"marker\"\n\tcase ConflictStragetyIgnore:\n\t\treturn \"ignore\"\n\tcase ConflictStragetyEmbrace:\n\t\treturn \"embrace\"\n\tdefault:\n\t\treturn \"unknown\"\n\t}\n}\n\n// ConflictStrategyFromString converts a string to a ConflictStrategy.\n// It it is not valid, ConflictStragetyUnknown is returned.\nfunc ConflictStrategyFromString(spec string) ConflictStrategy {\n\tswitch spec {\n\tcase \"marker\":\n\t\treturn ConflictStragetyMarker\n\tcase \"ignore\":\n\t\treturn ConflictStragetyIgnore\n\tcase \"embrace\":\n\t\treturn ConflictStragetyEmbrace\n\tdefault:\n\t\treturn ConflictStragetyUnknown\n\t}\n}\n\n// PinStats Handy structure to use during handleMerge to store the node info in which we merge\ntype PinStats struct {\n\tPinned, Explicit bool\n}\n\n// SyncOptions gives you the possibility to configure the sync algorithm.\ntype SyncOptions struct {\n\tConflictStrategy ConflictStrategy\n\tIgnoreDeletes bool\n\tIgnoreMoves bool\n\tMessage string\n\tReadOnlyFolders map[string]bool\n\tConflictStrategyPerFolder map[string]ConflictStrategy\n\n\tOnAdd func(newNd n.ModNode) bool\n\tOnRemove func(oldNd n.ModNode) bool\n\tOnMerge func(nd n.ModNode, isGet bool, ndPinStats *PinStats) bool\n\tOnConflict func(src, dst n.ModNode) bool\n}\n\nvar (\n\tdefaultSyncConfig = &SyncOptions{}\n)\n\ntype syncer struct {\n\tcfg *SyncOptions\n\tlkrSrc *c.Linker\n\tlkrDst *c.Linker\n}\n\nfunc (sy *syncer) add(src n.ModNode, srcParent, srcName string) error {\n\tvar newDstNode n.ModNode\n\tvar err error\n\n\tparentDir, err := sy.lkrDst.LookupDirectory(srcParent)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tswitch src.Type() {\n\tcase n.NodeTypeDirectory:\n\t\tnewDstNode, err = n.NewEmptyDirectory(\n\t\t\tsy.lkrDst,\n\t\t\tparentDir,\n\t\t\tsrcName,\n\t\t\tsrc.User(),\n\t\t\tsy.lkrDst.NextInode(),\n\t\t)\n\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif err := sy.lkrDst.StageNode(newDstNode); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tsrcDir, ok := src.(*n.Directory)\n\t\tif !ok {\n\t\t\treturn ie.ErrBadNode\n\t\t}\n\n\t\tchildren, err := srcDir.ChildrenSorted(sy.lkrSrc)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfor _, child := range children {\n\t\t\tchildModNode, ok := child.(n.ModNode)\n\t\t\tif !ok {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tif err := sy.add(childModNode, srcDir.Path(), child.Name()); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\tcase n.NodeTypeFile:\n\t\tnewDstFile := n.NewEmptyFile(\n\t\t\tparentDir,\n\t\t\tsrcName,\n\t\t\tsrc.User(),\n\t\t\tsy.lkrDst.NextInode(),\n\t\t)\n\n\t\tnewDstNode = newDstFile\n\n\t\tsrcFile, ok := src.(*n.File)\n\t\tif ok {\n\t\t\tnewDstFile.SetContent(sy.lkrDst, srcFile.ContentHash())\n\t\t\tnewDstFile.SetBackend(sy.lkrDst, srcFile.BackendHash())\n\t\t\tnewDstFile.SetSize(srcFile.Size())\n\t\t\tnewDstFile.SetCachedSize(srcFile.CachedSize())\n\t\t\tnewDstFile.SetKey(srcFile.Key())\n\t\t}\n\n\t\tif sy.cfg.OnAdd != nil {\n\t\t\tif !sy.cfg.OnAdd(newDstFile) {\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\n\t\tif err := parentDir.Add(sy.lkrDst, newDstFile); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn sy.lkrDst.StageNode(newDstNode)\n\tcase n.NodeTypeGhost:\n\t\t// skipping addition of a ghost\n\t\treturn nil\n\tdefault:\n\t\treturn fmt.Errorf(\"Unexpected node type in handleAdd\")\n\t}\n\n\treturn nil\n}\n\nfunc isReadOnly(folders map[string]bool, nodePaths ...string) bool {\n\tfor _, nodePath := range nodePaths {\n\t\tfor {\n\t\t\tif folders[nodePath] {\n\t\t\t\treturn true\n\t\t\t}\n\n\t\t\tnewNodePath := path.Dir(nodePath)\n\t\t\tif newNodePath == nodePath {\n\t\t\t\tbreak\n\t\t\t}\n\n\t\t\tnodePath = newNodePath\n\t\t}\n\t}\n\n\treturn false\n}\n\nfunc (sy *syncer) handleAdd(src n.ModNode) error {\n\tif isReadOnly(sy.cfg.ReadOnlyFolders, src.Path()) {\n\t\treturn nil\n\t}\n\n\tlog.Debugf(\"handling add: %s\", src.Path())\n\treturn sy.add(src, path.Dir(src.Path()), src.Name())\n}\n\nfunc (sy *syncer) handleMove(src, dst n.ModNode) error {\n\tif sy.cfg.IgnoreMoves {\n\t\treturn nil\n\t}\n\n\tif isReadOnly(sy.cfg.ReadOnlyFolders, src.Path(), dst.Path()) {\n\t\treturn nil\n\t}\n\n\tlog.Debugf(\"handling move: %s -> %s\", dst.Path(), src.Path())\n\tif _, err := c.Mkdir(sy.lkrDst, path.Dir(src.Path()), true); err != nil {\n\t\treturn err\n\t}\n\n\t// Move our node (dst) to the path determined by src.\n\treturn e.Wrapf(c.Move(sy.lkrDst, dst, src.Path()), \"move\")\n}\n\nfunc (sy *syncer) handleMissing(dst n.ModNode) error {\n\t// This is only called when a file in dst is missing on src.\n\t// No sync action is required.\n\tlog.Debugf(\"handling missing: %s\", dst.Path())\n\treturn nil\n}\n\nfunc (sy *syncer) handleRemove(dst n.ModNode) error {\n\tif sy.cfg.IgnoreDeletes {\n\t\treturn nil\n\t}\n\n\tif isReadOnly(sy.cfg.ReadOnlyFolders, dst.Path()) {\n\t\treturn nil\n\t}\n\n\tlog.Debugf(\"handling remove: %s\", dst.Path())\n\n\tif sy.cfg.OnRemove != nil {\n\t\tif !sy.cfg.OnRemove(dst) {\n\t\t\treturn nil\n\t\t}\n\t}\n\n\t_, _, err := c.Remove(sy.lkrDst, dst, true, true)\n\treturn err\n}\n\nfunc (sy *syncer) getConflictStrategy(nd n.ModNode) ConflictStrategy {\n\tcurr := nd.Path()\n\n\t// Shortcurt: If the per-folder feature is not used,\n\t// we can skip this whole loop below.\n\tif len(sy.cfg.ConflictStrategyPerFolder) == 0 {\n\t\treturn sy.cfg.ConflictStrategy\n\t}\n\n\tlog.Debugf(\"*** MAP %v\", sy.cfg.ConflictStrategyPerFolder)\n\n\tfor {\n\t\tcs, ok := sy.cfg.ConflictStrategyPerFolder[curr]\n\t\tif ok {\n\t\t\treturn cs\n\t\t}\n\n\t\tparent := path.Dir(curr)\n\t\tif parent == curr {\n\t\t\tbreak\n\t\t}\n\n\t\tcurr = parent\n\t}\n\n\t// No special strategy found for this folder\n\treturn sy.cfg.ConflictStrategy\n}\n\nfunc (sy *syncer) handleConflict(src, dst n.ModNode, srcMask, dstMask ChangeType) error {\n\tcs := sy.getConflictStrategy(dst)\n\n\tif cs == ConflictStragetyIgnore {\n\t\treturn nil\n\t}\n\n\tif cs == ConflictStragetyEmbrace {\n\t\treturn sy.handleMerge(src, dst, srcMask, dstMask)\n\t}\n\n\tif isReadOnly(sy.cfg.ReadOnlyFolders, src.Path(), dst.Path()) {\n\t\treturn nil\n\t}\n\n\tlog.Debugf(\"handling conflict: %s <-> %s\", src.Path(), dst.Path())\n\n\t// Find a path that we do not have yet.\n\t// stamp := time.Now().Format(time.RFC3339)\n\tconflictName := \"\"\n\tconflictNameTmpl := fmt.Sprintf(\"%s.conflict.%%d\", dst.Name())\n\n\t// Fix the unlikely case that there is already a node at the conflict path:\n\tfor tries := 0; tries < 100; tries++ {\n\t\tconflictName = fmt.Sprintf(conflictNameTmpl, tries)\n\t\tdstNd, err := sy.lkrDst.LookupNode(conflictName)\n\t\tif err != nil && !ie.IsNoSuchFileError(err) {\n\t\t\treturn err\n\t\t}\n\n\t\tif dstNd == nil {\n\t\t\tbreak\n\t\t}\n\t}\n\n\tdstDirname := path.Dir(dst.Path())\n\n\tif sy.cfg.OnConflict != nil {\n\t\tif !sy.cfg.OnConflict(src, dst) {\n\t\t\treturn nil\n\t\t}\n\t}\n\n\treturn sy.add(src, dstDirname, conflictName)\n}\n\nfunc (sy *syncer) handleMerge(src, dst n.ModNode, srcMask, dstMask ChangeType) error {\n\tif isReadOnly(sy.cfg.ReadOnlyFolders, src.Path(), dst.Path()) {\n\t\treturn nil\n\t}\n\n\tif src.Path() != dst.Path() {\n\t\t// Only move the file if it was only moved on the remote side.\n\t\tif srcMask&ChangeTypeMove != 0 && dstMask&ChangeTypeMove == 0 {\n\t\t\tif err := c.Move(sy.lkrDst, dst, src.Path()); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t}\n\n\tif dst.Type() == n.NodeTypeGhost {\n\t\t// Nothing to do. We removed the file on our side,\n\t\t// but it's still on the remote side. Good for them,\n\t\t// but keep it deleted here.\n\t\treturn nil\n\t}\n\n\t// If src did not change, there's no need to sync the content.\n\t// If src has no changes, we know that dst must have changes,\n\t// otherwise it would have been reported as conflict.\n\tif srcMask&ChangeTypeModify == 0 && srcMask&ChangeTypeAdd == 0 {\n\t\treturn nil\n\t}\n\n\tdstParent, err := n.ParentDirectory(sy.lkrDst, dst)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif err := dstParent.RemoveChild(sy.lkrDst, dst); err != nil {\n\t\treturn err\n\t}\n\n\tdstFile, ok := dst.(*n.File)\n\tif !ok {\n\t\treturn ie.ErrBadNode\n\t}\n\n\tsrcFile, ok := src.(*n.File)\n\tif !ok {\n\t\treturn ie.ErrBadNode\n\t}\n\n\toldDstPinStats := PinStats{false, false}\n\tisGet := true\n\tif sy.cfg.OnMerge != nil {\n\t\tsy.cfg.OnMerge(dst, isGet, &oldDstPinStats)\n\t}\n\n\tdstFile.SetContent(sy.lkrDst, srcFile.ContentHash())\n\tdstFile.SetBackend(sy.lkrDst, srcFile.BackendHash())\n\tdstFile.SetSize(srcFile.Size())\n\tdstFile.SetCachedSize(srcFile.CachedSize())\n\tdstFile.SetKey(srcFile.Key())\n\n\tif err := dstParent.Add(sy.lkrDst, dstFile); err != nil {\n\t\treturn err\n\t}\n\n\tif sy.cfg.OnMerge != nil {\n\t\tif !sy.cfg.OnMerge(dst, !isGet, &oldDstPinStats) {\n\t\t\treturn nil\n\t\t}\n\t}\n\n\treturn sy.lkrDst.StageNode(dstFile)\n}\n\nfunc (sy *syncer) handleTypeConflict(src, dst n.ModNode) error {\n\tlog.Debugf(\"handling type conflict: %s <-> %s\", src.Path(), dst.Path())\n\n\t// Simply do nothing.\n\treturn nil\n}\n\nfunc (sy *syncer) handleConflictNode(src n.ModNode) error {\n\tlog.Debugf(\"handling node conflict: %s\", src.Path())\n\n\t// We don't care for files on the other side named \"README.conflict.0\" e.g.\n\treturn nil\n}\n\n// Sync will synchronize the changes from `lkrSrc` to `lkrDst`,\n// according to the options set in `cfg`. This is atomic.\n// A new commit might be created with `message`, defaulting to a default message\n// when an empty string was given.\nfunc Sync(lkrSrc, lkrDst *c.Linker, cfg *SyncOptions) error {\n\tif cfg == nil {\n\t\tcfg = defaultSyncConfig\n\t}\n\n\tsyncer := &syncer{\n\t\tcfg: cfg,\n\t\tlkrSrc: lkrSrc,\n\t\tlkrDst: lkrDst,\n\t}\n\n\tresolver, err := newResolver(lkrSrc, lkrDst, nil, nil, syncer)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Make sure the complete sync goes through in one disk transaction.\n\treturn lkrDst.Atomic(func() (bool, error) {\n\t\t// This calls all the handleXXX() callbacks above.\n\t\tif err := resolver.resolve(); err != nil {\n\t\t\treturn true, err\n\t\t}\n\n\t\twasModified, err := lkrDst.HaveStagedChanges()\n\t\tif err != nil {\n\t\t\treturn true, err\n\t\t}\n\n\t\t// If something was changed, we should set the merge marker\n\t\t// and also create a new commit.\n\t\tif wasModified {\n\t\t\tsrcOwner, err := lkrSrc.Owner()\n\t\t\tif err != nil {\n\t\t\t\treturn true, err\n\t\t\t}\n\n\t\t\tsrcHead, err := lkrSrc.Head()\n\t\t\tif err != nil {\n\t\t\t\treturn true, err\n\t\t\t}\n\n\t\t\t// If something was changed, remember that we merged with src.\n\t\t\t// This avoids merging conflicting files a second time in the next resolve().\n\t\t\tif err := lkrDst.SetMergeMarker(srcOwner, srcHead.TreeHash()); err != nil {\n\t\t\t\treturn true, err\n\t\t\t}\n\n\t\t\tmessage := cfg.Message\n\t\t\tif message == \"\" {\n\t\t\t\tmessage = fmt.Sprintf(\"merge with »%s«\", srcOwner)\n\t\t\t}\n\n\t\t\tif err := lkrDst.MakeCommit(srcOwner, message); err != nil {\n\t\t\t\treturn true, err\n\t\t\t}\n\t\t}\n\n\t\treturn false, nil\n\t})\n}\n" }, { "path": "catfs/vcs/sync_test.go", "content": "package vcs\n\nimport (\n\t\"testing\"\n\n\tc \"github.com/sahib/brig/catfs/core\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// Create a file in src and check\n// that it's being synced to the dst side.\nfunc setupBasicSrcFile(t *testing.T, lkrSrc, lkrDst *c.Linker) {\n\tc.MustTouch(t, lkrSrc, \"/x.png\", 1)\n}\n\nfunc checkBasicSrcFile(t *testing.T, lkrSrc, lkrDst *c.Linker) {\n\txFile, err := lkrDst.LookupFile(\"/x.png\")\n\trequire.Nil(t, err)\n\trequire.Equal(t, xFile.Path(), \"/x.png\")\n\trequire.Equal(t, xFile.BackendHash(), h.TestDummy(t, 1))\n}\n\n////////\n\n// Only have the file on dst.\n// Nothing should happen, since no pair can be found.\nfunc setupBasicDstFile(t *testing.T, lkrSrc, lkrDst *c.Linker) {\n\tc.MustTouch(t, lkrDst, \"/x.png\", 1)\n}\n\nfunc checkBasicDstFile(t *testing.T, lkrSrc, lkrDst *c.Linker) {\n\txFile, err := lkrDst.LookupFile(\"/x.png\")\n\trequire.Nil(t, err)\n\trequire.Equal(t, xFile.Path(), \"/x.png\")\n\trequire.Equal(t, xFile.BackendHash(), h.TestDummy(t, 1))\n}\n\n////////\n\n// Create the same file on both sides with the same content.\nfunc setupBasicBothNoConflict(t *testing.T, lkrSrc, lkrDst *c.Linker) {\n\tc.MustTouch(t, lkrSrc, \"/x.png\", 1)\n\tc.MustTouch(t, lkrDst, \"/x.png\", 1)\n}\n\nfunc checkBasicBothNoConflict(t *testing.T, lkrSrc, lkrDst *c.Linker) {\n\txSrcFile, err := lkrSrc.LookupFile(\"/x.png\")\n\trequire.Nil(t, err)\n\trequire.Equal(t, xSrcFile.Path(), \"/x.png\")\n\trequire.Equal(t, xSrcFile.BackendHash(), h.TestDummy(t, 1))\n\n\txDstFile, err := lkrDst.LookupFile(\"/x.png\")\n\trequire.Nil(t, err)\n\trequire.Equal(t, xDstFile.Path(), \"/x.png\")\n\trequire.Equal(t, xDstFile.BackendHash(), h.TestDummy(t, 1))\n}\n\n////////\n\n// Create the same file on both sides with different content.\n// This should result in a conflict, resulting in conflict file.\nfunc setupBasicBothConflict(t *testing.T, lkrSrc, lkrDst *c.Linker) {\n\tc.MustTouch(t, lkrSrc, \"/x.png\", 42)\n\tc.MustTouch(t, lkrDst, \"/x.png\", 23)\n}\n\nfunc checkBasicBothConflict(t *testing.T, lkrSrc, lkrDst *c.Linker) {\n\txSrcFile, err := lkrSrc.LookupFile(\"/x.png\")\n\trequire.Nil(t, err)\n\trequire.Equal(t, xSrcFile.Path(), \"/x.png\")\n\trequire.Equal(t, xSrcFile.BackendHash(), h.TestDummy(t, 42))\n\n\txDstFile, err := lkrDst.LookupFile(\"/x.png\")\n\trequire.Nil(t, err)\n\trequire.Equal(t, xDstFile.Path(), \"/x.png\")\n\trequire.Equal(t, xDstFile.BackendHash(), h.TestDummy(t, 23))\n\n\txConflictFile, err := lkrDst.LookupFile(\"/x.png.conflict.0\")\n\trequire.Nil(t, err)\n\trequire.Equal(t, xConflictFile.Path(), \"/x.png.conflict.0\")\n\trequire.Equal(t, xConflictFile.BackendHash(), h.TestDummy(t, 42))\n}\n\n////////\n\nfunc setupBasicRemove(t *testing.T, lkrSrc, lkrDst *c.Linker) {\n\t// Create x.png on src and remove it after one commit:\n\txFile := c.MustTouch(t, lkrSrc, \"/x.png\", 42)\n\tc.MustCommit(t, lkrSrc, \"who let the x out\")\n\tc.MustRemove(t, lkrSrc, xFile)\n\n\t// Create the same file on dst:\n\tc.MustTouch(t, lkrDst, \"/x.png\", 42)\n}\n\nfunc checkBasicRemove(t *testing.T, lkrSrc, lkrDst *c.Linker) {\n\txDstFile, err := lkrDst.LookupGhost(\"/x.png\")\n\trequire.Nil(t, err)\n\trequire.Equal(t, xDstFile.Path(), \"/x.png\")\n}\n\n////////\n\nfunc setupBasicSrcMove(t *testing.T, lkrSrc, lkrDst *c.Linker) {\n\t// Create x.png on src and remove it after one commit:\n\txFile := c.MustTouch(t, lkrSrc, \"/x.png\", 42)\n\tc.MustCommit(t, lkrSrc, \"who let the x out\")\n\tc.MustMove(t, lkrSrc, xFile, \"/y.png\")\n\n\t// Create the same file on dst:\n\tc.MustTouch(t, lkrDst, \"/x.png\", 42)\n}\n\nfunc checkBasicSrcMove(t *testing.T, lkrSrc, lkrDst *c.Linker) {\n\txDstGhost, err := lkrDst.LookupGhost(\"/x.png\")\n\trequire.Nil(t, err)\n\n\trequire.Equal(t, xDstGhost.Path(), \"/x.png\")\n\trequire.Equal(t, xDstGhost.BackendHash(), h.TestDummy(t, 42))\n\n\tyDstFile, err := lkrDst.LookupFile(\"/y.png\")\n\trequire.Nil(t, err)\n\n\trequire.Equal(t, yDstFile.Path(), \"/y.png\")\n\trequire.Equal(t, yDstFile.BackendHash(), h.TestDummy(t, 42))\n}\n\n////////\n\nfunc setupEdgeMoveDirAndModifyChild(t *testing.T, lkrSrc, lkrDst *c.Linker) {\n\t// Syncing recursive empty dirs require detecting and adding them recursive.\n\t// This was buggy before, so prevent it from happening again.\n\tc.MustMkdir(t, lkrSrc, \"/a\")\n\tc.MustMkdir(t, lkrDst, \"/a\")\n\tc.MustCommit(t, lkrSrc, \"added dirs src\")\n\tc.MustCommit(t, lkrDst, \"added dirs dst\")\n}\n\n////////\n\nfunc setupEdgeEmptyDir(t *testing.T, lkrSrc, lkrDst *c.Linker) {\n\t// Syncing recursive empty dirs require detecting and adding them recursive.\n\t// This was buggy before, so prevent it from happening again.\n\tc.MustMkdir(t, lkrSrc, \"/empty/sub/blub\")\n}\n\nfunc checkEdgeEmptyDir(t *testing.T, lkrSrc, lkrDst *c.Linker) {\n\tdir, err := lkrDst.LookupDirectory(\"/empty/sub/blub\")\n\trequire.Nil(t, err)\n\trequire.Equal(t, dir.Path(), \"/empty/sub/blub\")\n}\n\nfunc TestSync(t *testing.T) {\n\tt.Parallel()\n\n\ttcs := []struct {\n\t\tname string\n\t\tsetup func(t *testing.T, lkrSrc, lkrDst *c.Linker)\n\t\tcheck func(t *testing.T, lkrSrc, lkrDst *c.Linker)\n\t}{\n\t\t{\n\t\t\tname: \"basic-src-file\",\n\t\t\tsetup: setupBasicSrcFile,\n\t\t\tcheck: checkBasicSrcFile,\n\t\t}, {\n\t\t\tname: \"basic-dst-file\",\n\t\t\tsetup: setupBasicDstFile,\n\t\t\tcheck: checkBasicDstFile,\n\t\t}, {\n\t\t\tname: \"basic-both-file-no-conflict\",\n\t\t\tsetup: setupBasicBothNoConflict,\n\t\t\tcheck: checkBasicBothNoConflict,\n\t\t}, {\n\t\t\tname: \"basic-both-file-conflict\",\n\t\t\tsetup: setupBasicBothConflict,\n\t\t\tcheck: checkBasicBothConflict,\n\t\t}, {\n\t\t\tname: \"basic-src-remove\",\n\t\t\tsetup: setupBasicRemove,\n\t\t\tcheck: checkBasicRemove,\n\t\t}, {\n\t\t\tname: \"basic-src-move\",\n\t\t\tsetup: setupBasicSrcMove,\n\t\t\tcheck: checkBasicSrcMove,\n\t\t}, {\n\t\t\tname: \"edge-empty-dir\",\n\t\t\tsetup: setupEdgeEmptyDir,\n\t\t\tcheck: checkEdgeEmptyDir,\n\t\t},\n\t}\n\n\tfor _, tc := range tcs {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tt.Parallel()\n\n\t\t\tc.WithLinkerPair(t, func(lkrSrc, lkrDst *c.Linker) {\n\t\t\t\ttc.setup(t, lkrSrc, lkrDst)\n\t\t\t\t// sync requires that all changes are committed.\n\t\t\t\tc.MustCommitIfPossible(t, lkrDst, \"setup dst\")\n\t\t\t\tc.MustCommitIfPossible(t, lkrSrc, \"setup src\")\n\n\t\t\t\tif err := Sync(lkrSrc, lkrDst, nil); err != nil {\n\t\t\t\t\tt.Fatalf(\"sync failed: %v\", err)\n\t\t\t\t}\n\n\t\t\t\ttc.check(t, lkrSrc, lkrDst)\n\t\t\t})\n\t\t})\n\t}\n}\n\nfunc TestSyncMergeMarker(t *testing.T) {\n\tc.WithLinkerPair(t, func(lkrSrc, lkrDst *c.Linker) {\n\t\tc.MustTouchAndCommit(t, lkrSrc, \"/x.png\", 1)\n\t\tc.MustTouchAndCommit(t, lkrDst, \"/y.png\", 2)\n\n\t\tif err := Sync(lkrSrc, lkrDst, nil); err != nil {\n\t\t\tt.Fatalf(\"sync failed: %v\", err)\n\t\t}\n\n\t\tdstHead, err := lkrDst.Head()\n\t\trequire.Nil(t, err)\n\n\t\tsrcHead, err := lkrSrc.Head()\n\t\trequire.Nil(t, err)\n\n\t\tmergeUser, mergeHash := dstHead.MergeMarker()\n\t\trequire.Equal(t, mergeUser, \"src\")\n\t\trequire.Equal(t, mergeHash, srcHead.TreeHash())\n\n\t\tc.MustTouch(t, lkrSrc, \"/a.png\", 3)\n\t\tc.MustTouch(t, lkrDst, \"/b.png\", 4)\n\n\t\tdiff, err := MakeDiff(lkrSrc, lkrDst, nil, nil, nil)\n\t\trequire.Nil(t, err)\n\n\t\trequire.Empty(t, diff.Conflict)\n\t\trequire.Empty(t, diff.Ignored)\n\t\trequire.Empty(t, diff.Merged)\n\t\trequire.Empty(t, diff.Removed)\n\n\t\trequire.Len(t, diff.Added, 1)\n\t\trequire.Len(t, diff.Missing, 2)\n\n\t\trequire.Equal(t, diff.Added[0].Path(), \"/a.png\")\n\t\trequire.Equal(t, diff.Missing[0].Path(), \"/b.png\")\n\t\trequire.Equal(t, diff.Missing[1].Path(), \"/y.png\")\n\t})\n}\n\nfunc TestSyncConflictMergeMarker(t *testing.T) {\n\tc.WithLinkerPair(t, func(lkrSrc, lkrDst *c.Linker) {\n\t\tc.MustTouchAndCommit(t, lkrSrc, \"/x.png\", 1)\n\t\tc.MustTouchAndCommit(t, lkrDst, \"/x.png\", 2)\n\n\t\tif err := Sync(lkrSrc, lkrDst, nil); err != nil {\n\t\t\tt.Fatalf(\"sync failed: %v\", err)\n\t\t}\n\n\t\tdstHead, err := lkrDst.Head()\n\t\trequire.Nil(t, err)\n\n\t\tsrcHead, err := lkrSrc.Head()\n\t\trequire.Nil(t, err)\n\n\t\tmergeUser, mergeHash := dstHead.MergeMarker()\n\t\trequire.Equal(t, mergeUser, \"src\")\n\t\trequire.Equal(t, mergeHash, srcHead.TreeHash())\n\n\t\tc.MustTouch(t, lkrSrc, \"/a.png\", 3)\n\t\tc.MustTouch(t, lkrDst, \"/a.png\", 4)\n\n\t\tdiff, err := MakeDiff(lkrSrc, lkrDst, nil, nil, nil)\n\t\trequire.Nil(t, err)\n\n\t\trequire.Len(t, diff.Merged, 0)\n\t\trequire.Len(t, diff.Ignored, 1)\n\t\trequire.Len(t, diff.Conflict, 1)\n\n\t\trequire.Empty(t, diff.Moved)\n\t\trequire.Empty(t, diff.Missing)\n\t\trequire.Empty(t, diff.Added)\n\t\trequire.Empty(t, diff.Removed)\n\n\t\t// a.png is new and will conflict therefore.\n\t\trequire.Equal(t, diff.Conflict[0].Dst.Path(), \"/a.png\")\n\t\trequire.Equal(t, diff.Conflict[0].Src.Path(), \"/a.png\")\n\n\t\t// The previously created conflict file should count as missing.\n\t\trequire.Equal(t, diff.Ignored[0].Path(), \"/x.png.conflict.0\")\n\t})\n}\n\nfunc TestSyncTwiceWithMovedFile(t *testing.T) {\n\tc.WithLinkerPair(t, func(lkrAli, lkrBob *c.Linker) {\n\t\taliNd, _ := c.MustTouchAndCommit(t, lkrAli, \"/ali-file\", 1)\n\t\tbobNd, _ := c.MustTouchAndCommit(t, lkrBob, \"/bob-file\", 2)\n\n\t\trequire.Nil(t, Sync(lkrAli, lkrBob, nil))\n\t\trequire.Nil(t, Sync(lkrBob, lkrAli, nil))\n\n\t\tc.MustMove(t, lkrAli, aliNd, \"/bali-bile\")\n\t\tc.MustMove(t, lkrBob, bobNd, \"/blob-lile\")\n\t\tc.MustCommit(t, lkrAli, \"moved file\")\n\n\t\tdiff, err := MakeDiff(lkrBob, lkrAli, nil, nil, nil)\n\t\trequire.Nil(t, err)\n\n\t\trequire.Len(t, diff.Added, 0)\n\t\trequire.Len(t, diff.Removed, 0)\n\t\trequire.Len(t, diff.Moved, 2)\n\t})\n}\n\nfunc TestSyncConflictStrategyEmbrace(t *testing.T) {\n\tc.WithLinkerPair(t, func(lkrSrc, lkrDst *c.Linker) {\n\t\tc.MustTouchAndCommit(t, lkrSrc, \"/x.png\", 1)\n\t\tc.MustTouchAndCommit(t, lkrDst, \"/x.png\", 2)\n\n\t\tcfg := &SyncOptions{\n\t\t\tConflictStrategy: ConflictStragetyEmbrace,\n\t\t}\n\n\t\tdiff, err := MakeDiff(lkrSrc, lkrDst, nil, nil, cfg)\n\t\trequire.Nil(t, err)\n\n\t\trequire.Len(t, diff.Conflict, 1)\n\t\trequire.Empty(t, diff.Merged)\n\t\trequire.Empty(t, diff.Ignored)\n\t\trequire.Empty(t, diff.Moved)\n\t\trequire.Empty(t, diff.Missing)\n\t\trequire.Empty(t, diff.Added)\n\t\trequire.Empty(t, diff.Removed)\n\n\t\trequire.Nil(t, Sync(lkrSrc, lkrDst, cfg))\n\n\t\tsrcX, err := lkrSrc.LookupFile(\"/x.png\")\n\t\trequire.Nil(t, err)\n\t\tdstX, err := lkrDst.LookupFile(\"/x.png\")\n\t\trequire.Nil(t, err)\n\n\t\trequire.Equal(t, srcX.ContentHash(), dstX.ContentHash())\n\t})\n}\n\nfunc TestSyncReadOnlyFolders(t *testing.T) {\n\tc.WithLinkerPair(t, func(lkrSrc, lkrDst *c.Linker) {\n\t\t// Create a file on alice' side:\n\t\tc.MustTouchAndCommit(t, lkrSrc, \"/public/x.png\", 1)\n\t\tcfg := &SyncOptions{\n\t\t\tReadOnlyFolders: map[string]bool{\n\t\t\t\t\"/public\": true,\n\t\t\t},\n\t\t}\n\n\t\t// Sync without a config - this is \"bob's\" side.\n\t\t// (he does not have any read-only folders)\n\t\trequire.Nil(t, Sync(lkrSrc, lkrDst, nil))\n\n\t\t// Both alice and bob should have the same file/content:\n\t\tsrcX, err := lkrSrc.LookupFile(\"/public/x.png\")\n\t\trequire.Nil(t, err)\n\t\tdstX, err := lkrDst.LookupFile(\"/public/x.png\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, srcX.ContentHash(), dstX.ContentHash())\n\n\t\t// bob modifies /public/x.png\n\t\tc.MustModify(t, lkrDst, dstX, 2)\n\t\tdstX, err = lkrDst.LookupFile(\"/public/x.png\")\n\t\trequire.Nil(t, err)\n\n\t\t// let alice sync back the change of bob:\n\t\trequire.Nil(t, Sync(lkrDst, lkrSrc, cfg))\n\n\t\tsrcX, err = lkrSrc.LookupFile(\"/public/x.png\")\n\t\trequire.Nil(t, err)\n\n\t\trequire.NotEqual(t, srcX.ContentHash(), dstX.ContentHash())\n\t\trequire.Equal(t, srcX.ContentHash(), h.TestDummy(t, byte(1)))\n\t})\n}\n" }, { "path": "catfs/vcs/undelete.go", "content": "package vcs\n\nimport (\n\t\"fmt\"\n\n\tc \"github.com/sahib/brig/catfs/core\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\tn \"github.com/sahib/brig/catfs/nodes\"\n)\n\n// Undelete tries to recover the node pointed to by `root`.\n// The node must be a ghost, otherwise we will error out.\nfunc Undelete(lkr *c.Linker, root string) error {\n\tnd, err := lkr.LookupModNode(root)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif nd.Type() != n.NodeTypeGhost {\n\t\treturn fmt.Errorf(\"%s is not a deleted file: %v\", root, err)\n\t}\n\n\tcmt, err := lkr.Status()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tvar origNd n.ModNode\n\n\t// Walk to the last point in history where the ghost\n\t// was either removed or moved. In theory it could have been\n\t// modified or added in between, but that would mean that\n\t// someone played around with the graph.\n\twalker := NewHistoryWalker(lkr, cmt, nd)\n\tfor walker.Next() {\n\t\tstate := walker.State()\n\t\ttyp := state.Curr.Type()\n\t\tif typ != n.NodeTypeGhost {\n\t\t\tcontinue\n\t\t}\n\n\t\tif state.Mask&ChangeTypeRemove == 0 {\n\t\t\tcontinue\n\t\t}\n\n\t\tif state.Mask&ChangeTypeMove > 0 {\n\t\t\tcontinue\n\t\t}\n\n\t\t// We know now that we're on the ghost was added after deleting\n\t\t// or removing the file. Now go one back to reach the actual node.\n\t\tif !walker.Next() {\n\t\t\tbreak\n\t\t}\n\n\t\torigNd = walker.State().Curr\n\t\tbreak\n\t}\n\n\tif origNd == nil {\n\t\treturn fmt.Errorf(\"could not find a state where this file was not deleted\")\n\t}\n\n\t// Do the actual recovery. Handle the case where we are undeleting a\n\t// whole directory tree with possibly empty directories inside.\n\treturn lkr.Atomic(func() (bool, error) {\n\t\treturn true, n.Walk(lkr, origNd, true, func(child n.Node) error {\n\t\t\tswitch child.Type() {\n\t\t\tcase n.NodeTypeDirectory:\n\t\t\t\tdir, ok := child.(*n.Directory)\n\t\t\t\tif !ok {\n\t\t\t\t\treturn ie.ErrBadNode\n\t\t\t\t}\n\n\t\t\t\t// Create empty directories manually,\n\t\t\t\t// all other directories will be created implicitly:\n\t\t\t\tif dir.NChildren() == 0 {\n\t\t\t\t\t_, err := c.Mkdir(lkr, dir.Path(), true)\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\tcase n.NodeTypeFile:\n\t\t\t\tfile, ok := child.(*n.File)\n\t\t\t\tif !ok {\n\t\t\t\t\treturn ie.ErrBadNode\n\t\t\t\t}\n\n\t\t\t\t// Stage that old state:\n\t\t\t\t_, err := c.StageFromFileNode(lkr, file)\n\t\t\t\treturn err\n\t\t\t}\n\t\t\treturn nil\n\t\t})\n\t})\n}\n" }, { "path": "client/.gitignore", "content": "discovery/\n" }, { "path": "client/client.go", "content": "package client\n\nimport (\n\t\"context\"\n\t\"net\"\n\n\t\"github.com/sahib/brig/server/capnp\"\n\t\"github.com/sahib/brig/util\"\n\t\"zombiezen.com/go/capnproto2/rpc\"\n)\n\n// Client is a helper API that implements the rpc interface to brig and makes\n// all data easily accessible from Go. Note that this layer is needed, so we\n// could later support other languages.\ntype Client struct {\n\tctx context.Context\n\tconn *rpc.Conn\n\trawConn net.Conn\n\tapi capnp.API\n}\n\nfunc connFromURL(s string) (net.Conn, error) {\n\tscheme, addr, err := util.URLToSchemeAndAddr(s)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn net.Dial(scheme, addr)\n}\n\n// Dial will attempt to connect to brigd under the specified port\nfunc Dial(ctx context.Context, daemonURL string) (*Client, error) {\n\trawConn, err := connFromURL(daemonURL)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\ttransport := rpc.StreamTransport(rawConn)\n\tconn := rpc.NewConn(\n\t\ttransport,\n\t\trpc.ConnLog(nil),\n\t\trpc.SendBufferSize(128),\n\t)\n\tapi := capnp.API{Client: conn.Bootstrap(ctx)}\n\n\treturn &Client{\n\t\tctx: ctx,\n\t\trawConn: rawConn,\n\t\tconn: conn,\n\t\tapi: api,\n\t}, nil\n}\n\n// LocalAddr return info about the local addr\nfunc (cl *Client) LocalAddr() net.Addr {\n\treturn cl.rawConn.LocalAddr()\n}\n\n// RemoteAddr return info about the remote addr\nfunc (cl *Client) RemoteAddr() net.Addr {\n\treturn cl.rawConn.RemoteAddr()\n}\n\n// Close will close the connection from the client side\nfunc (cl *Client) Close() error {\n\treturn cl.conn.Close()\n}\n" }, { "path": "client/clienttest/daemon.go", "content": "package clienttest\n\nimport (\n\t\"context\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"time\"\n\n\t\"github.com/sahib/brig/client\"\n\t\"github.com/sahib/brig/repo\"\n\t\"github.com/sahib/brig/server\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// StartDaemon starts a new daemon with user `name`, using backend defined by\n// `backendName` and, if the backend is IPFS, uses the IPFS repository at\n// `ipfsPath`. The resulting server should be closed after use and the\n// temporary directory where all data resides should be removed.\nfunc StartDaemon(name, backendName, ipfsPath string) (*server.Server, error) {\n\trepoPath, err := ioutil.TempDir(\"\", \"brig-client-repo\")\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdaemonURL := \"unix:\" + filepath.Join(repoPath, \"brig.socket\")\n\tif err := repo.Init(repo.InitOptions{\n\t\tBaseFolder: repoPath,\n\t\tOwner: name,\n\t\tBackendName: backendName,\n\t\tDaemonURL: daemonURL,\n\t}); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif backendName == \"httpipfs\" {\n\t\tif err := repo.OverwriteConfigKey(\n\t\t\trepoPath,\n\t\t\t\"daemon.ipfs_path_or_url\",\n\t\t\tipfsPath,\n\t\t); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tsrv, err := server.BootServer(repoPath, daemonURL)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tgo func() {\n\t\tif err := srv.Serve(); err != nil {\n\t\t\tlog.WithError(err).Warnf(\"failed to serve\")\n\t\t}\n\t}()\n\n\t// give some time for startup:\n\ttime.Sleep(500 * time.Millisecond)\n\treturn srv, nil\n}\n\n// WithDaemon calls `fn` with a readily setup daemon client. `name` is the user.\nfunc WithDaemon(name string, fn func(ctl *client.Client) error) error {\n\tsrv, err := StartDaemon(name, \"mock\", \"\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdefer func() {\n\t\t// Somehow there is race condition between\n\t\t// srv.Close() from the defer at the very end\n\t\t// os.RemoveAll(repoPath).\n\t\t// Theoretically, `go` should have closed server\n\t\t// but in practice I see that repoPath is removed\n\t\t// before server had a chance to close the DB\n\t\t// and I see complains in log about DB.Close\n\t\t// I introduce this time delay as a crude hack\n\t\ttime.Sleep(100 * time.Millisecond)\n\t\tos.RemoveAll(srv.RepoPath())\n\t}()\n\tdefer srv.Close()\n\n\tctl, err := client.Dial(context.Background(), srv.DaemonURL())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdefer ctl.Close()\n\n\treturn fn(ctl)\n}\n\n// WithDaemonPair calls `fn` with two readily setup daemon clients.\n// `nameA` and `nameB` are the respective names.\nfunc WithDaemonPair(nameA, nameB string, fn func(ctlA, ctlB *client.Client) error) error {\n\treturn WithDaemon(nameA, func(ctlA *client.Client) error {\n\t\treturn WithDaemon(nameB, func(ctlB *client.Client) error {\n\t\t\taliWhoami, err := ctlA.Whoami()\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tbobWhoami, err := ctlB.Whoami()\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\t// add bob to ali as remote\n\t\t\tif err := ctlA.RemoteAddOrUpdate(client.Remote{\n\t\t\t\tName: nameB,\n\t\t\t\tFingerprint: bobWhoami.Fingerprint,\n\t\t\t}); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\t// add ali to bob as remote\n\t\t\tif err := ctlB.RemoteAddOrUpdate(client.Remote{\n\t\t\t\tName: nameA,\n\t\t\t\tFingerprint: aliWhoami.Fingerprint,\n\t\t\t}); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\treturn fn(ctlA, ctlB)\n\t\t})\n\t})\n}\n" }, { "path": "client/fs_cmds.go", "content": "package client\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/sahib/brig/backend/httpipfs\"\n\t\"github.com/sahib/brig/catfs/mio\"\n\t\"github.com/sahib/brig/server/capnp\"\n\th \"github.com/sahib/brig/util/hashlib\"\n)\n\n// StatInfo gives information about a file or directory\n// similar to the normal stat(2) call on POSIX.\ntype StatInfo struct {\n\tPath string\n\tUser string\n\tSize uint64\n\tCachedSize int64\n\tInode uint64\n\tIsDir bool\n\tIsRaw bool\n\tDepth int\n\tModTime time.Time\n\tIsPinned bool\n\tIsExplicit bool\n\tTreeHash h.Hash\n\tContentHash h.Hash\n\tBackendHash h.Hash\n\tKey []byte\n\tHint Hint\n}\n\nfunc convertHash(hashBytes []byte, err error) (h.Hash, error) {\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn h.Cast(hashBytes)\n}\n\nfunc convertCapStatInfo(capInfo *capnp.StatInfo) (*StatInfo, error) {\n\tinfo := &StatInfo{}\n\n\tpath, err := capInfo.Path()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tuser, err := capInfo.User()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\ttreeHash, err := convertHash(capInfo.TreeHash())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcontentHash, err := convertHash(capInfo.ContentHash())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tbackendHash, err := convertHash(capInfo.BackendHash())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tkey, err := capInfo.Key()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tmodTimeData, err := capInfo.ModTime()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := info.ModTime.UnmarshalText([]byte(modTimeData)); err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapHint, err := capInfo.Hint()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\thint, err := convertCapHint(capHint)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tinfo.Path = path\n\tinfo.User = user\n\tinfo.Size = capInfo.Size()\n\tinfo.CachedSize = capInfo.CachedSize()\n\tinfo.Inode = capInfo.Inode()\n\tinfo.IsDir = capInfo.IsDir()\n\tinfo.IsRaw = capInfo.IsRaw()\n\tinfo.IsPinned = capInfo.IsPinned()\n\tinfo.IsExplicit = capInfo.IsExplicit()\n\tinfo.Depth = int(capInfo.Depth())\n\n\tinfo.TreeHash = treeHash\n\tinfo.ContentHash = contentHash\n\tinfo.BackendHash = backendHash\n\tinfo.Key = key\n\tinfo.Hint = *hint\n\treturn info, nil\n}\n\n// List will list all nodes beneath and including `root` up to `maxDepth`.\nfunc (cl *Client) List(root string, maxDepth int) ([]StatInfo, error) {\n\tcall := cl.api.List(cl.ctx, func(p capnp.FS_list_Params) error {\n\t\tp.SetMaxDepth(int32(maxDepth))\n\t\treturn p.SetRoot(root)\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tresults := []StatInfo{}\n\tstatList, err := result.Entries()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor idx := 0; idx < statList.Len(); idx++ {\n\t\tcapInfo := statList.At(idx)\n\t\tinfo, err := convertCapStatInfo(&capInfo)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tresults = append(results, *info)\n\t}\n\n\treturn results, err\n}\n\n// Stage will add a new node at `repoPath` with the contents of `localPath`.\nfunc (cl *Client) Stage(localPath, repoPath string) error {\n\tcall := cl.api.Stage(cl.ctx, func(p capnp.FS_stage_Params) error {\n\t\tif err := p.SetRepoPath(repoPath); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn p.SetLocalPath(localPath)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// StageFromReader will create a new node at `repoPath` from the contents of `r`.\nfunc (cl *Client) StageFromReader(repoPath string, r io.Reader) error {\n\tcall := cl.api.StageFromStream(cl.ctx, func(p capnp.FS_stageFromStream_Params) error {\n\t\treturn p.SetRepoPath(repoPath)\n\t})\n\n\t// NOTE: Promise pipelining happens here,\n\t// cb might not have been returned yet by the server.\n\t// We can still use it, since Cap'n Proto returns a promise here.\n\t// First network call happens only at the first Struct() call.\n\tstream := call.Stream()\n\n\t// relative large buffer to minimize Cap'n Proto overhead even further.\n\tbuf := make([]byte, 128*1024)\n\tchunkIdx, blockCheck := 0, 1\n\n\tfor {\n\t\tisEOF := false\n\t\tn, err := io.ReadFull(r, buf)\n\t\tif err != nil {\n\t\t\tif err == io.EOF || err == io.ErrUnexpectedEOF {\n\t\t\t\tisEOF = true\n\t\t\t} else {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\tif n > 0 {\n\t\t\tpromise := stream.SendChunk(cl.ctx, func(params capnp.FS_StageStream_sendChunk_Params) error {\n\t\t\t\treturn params.SetChunk(buf[:n])\n\t\t\t})\n\n\t\t\t// Assumption here: If transfer fails it will fail in the first few blocks.\n\t\t\t// For the rest of the block we can skip error checks on most blocks.\n\t\t\tif chunkIdx%blockCheck == 0 {\n\t\t\t\tif _, err := promise.Struct(); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\n\t\t\t\tif blockCheck < 128 {\n\t\t\t\t\tblockCheck *= 2\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tchunkIdx++\n\t\t}\n\n\t\tif isEOF {\n\t\t\tbreak\n\t\t}\n\t}\n\n\t// Tell the server side that we're done sending chunks and that the data\n\t// should be already staged.\n\t_, err := stream.Done(cl.ctx, nil).Struct()\n\treturn err\n}\n\n// Cat outputs the contents of the node at `path`.\n// The node must be a file.\nfunc (cl *Client) Cat(path string, offline bool) (io.ReadCloser, error) {\n\tcall := cl.api.Cat(cl.ctx, func(p capnp.FS_cat_Params) error {\n\t\tp.SetOffline(offline)\n\t\treturn p.SetPath(path)\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tport := result.Port()\n\tconn, err := net.Dial(\"tcp\", fmt.Sprintf(\"localhost:%d\", port))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn conn, nil\n}\n\n// CatOnClient is like Cat, but will fetch the stream directly from IPFS\n// and decode it on the client side. This is usually faster than other way round.\nfunc (cl *Client) CatOnClient(path string, offline bool, w io.Writer) error {\n\tinfo, err := cl.Stat(path)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tipfsPathOrMultiaddr, err := cl.ConfigGet(\"daemon.ipfs_path_or_url\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif ipfsPathOrMultiaddr == \"\" {\n\t\treturn fmt.Errorf(\"no IPFS path or URL found - is this repo using IPFS?\")\n\t}\n\n\tif offline {\n\t\tisCached, err := cl.IsCached(path)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif !isCached {\n\t\t\treturn fmt.Errorf(\"not cached\")\n\t\t}\n\t}\n\n\tnd, err := httpipfs.NewNode(\n\t\tipfsPathOrMultiaddr,\n\t\t\"\",\n\t\thttpipfs.WithNoLogging(),\n\t)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdefer nd.Close()\n\n\tipfsStream, err := nd.Cat(info.BackendHash)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdefer ipfsStream.Close()\n\n\tstream, err := mio.NewOutStream(ipfsStream, info.IsRaw, info.Key)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t_, err = io.Copy(os.Stdout, stream)\n\treturn err\n}\n\n// Tar outputs a tar archive with the contents of `path`.\n// `path` can be either a file or directory.\nfunc (cl *Client) Tar(path string, offline bool) (io.ReadCloser, error) {\n\tcall := cl.api.Tar(cl.ctx, func(p capnp.FS_tar_Params) error {\n\t\tp.SetOffline(offline)\n\t\treturn p.SetPath(path)\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tport := result.Port()\n\tconn, err := net.Dial(\"tcp\", fmt.Sprintf(\"localhost:%d\", port))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn conn, nil\n}\n\n// Mkdir creates a new empty directory at `path`, possibly creating\n// intermediate directories if `createParents` is set.\nfunc (cl *Client) Mkdir(path string, createParents bool) error {\n\tcall := cl.api.Mkdir(cl.ctx, func(p capnp.FS_mkdir_Params) error {\n\t\tp.SetCreateParents(createParents)\n\t\treturn p.SetPath(path)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// Remove removes the node at `path`.\n// Directories are removed recursively.\nfunc (cl *Client) Remove(path string) error {\n\tcall := cl.api.Remove(cl.ctx, func(p capnp.FS_remove_Params) error {\n\t\treturn p.SetPath(path)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// Move moves the node at `srcPath` to `dstPath`.\nfunc (cl *Client) Move(srcPath, dstPath string) error {\n\tcall := cl.api.Move(cl.ctx, func(p capnp.FS_move_Params) error {\n\t\tif err := p.SetSrcPath(srcPath); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn p.SetDstPath(dstPath)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// Copy copies the node at `srcPath` to `dstPath`.\nfunc (cl *Client) Copy(srcPath, dstPath string) error {\n\tcall := cl.api.Copy(cl.ctx, func(p capnp.FS_copy_Params) error {\n\t\tif err := p.SetSrcPath(srcPath); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn p.SetDstPath(dstPath)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// Pin sets an explicit pin on the node at `path`.\nfunc (cl *Client) Pin(path string) error {\n\tcall := cl.api.Pin(cl.ctx, func(p capnp.FS_pin_Params) error {\n\t\treturn p.SetPath(path)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// Unpin removes an explicit pin at the node at `path`.\nfunc (cl *Client) Unpin(path string) error {\n\tcall := cl.api.Unpin(cl.ctx, func(p capnp.FS_unpin_Params) error {\n\t\treturn p.SetPath(path)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// Repin schedules a repinning operation\nfunc (cl *Client) Repin(root string) error {\n\tcall := cl.api.Repin(cl.ctx, func(p capnp.FS_repin_Params) error {\n\t\treturn p.SetPath(root)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// Stat gives detailed information about the node at `path`.\nfunc (cl *Client) Stat(path string) (*StatInfo, error) {\n\tcall := cl.api.Stat(cl.ctx, func(p capnp.FS_stat_Params) error {\n\t\treturn p.SetPath(path)\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapInfo, err := result.Info()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn convertCapStatInfo(&capInfo)\n}\n\n// Touch creates a new empty file at `path`.\nfunc (cl *Client) Touch(path string) error {\n\tcall := cl.api.Touch(cl.ctx, func(p capnp.FS_touch_Params) error {\n\t\treturn p.SetPath(path)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// Exists tells us if a file at `path` exists.\nfunc (cl *Client) Exists(path string) (bool, error) {\n\tcall := cl.api.Exists(cl.ctx, func(p capnp.FS_exists_Params) error {\n\t\treturn p.SetPath(path)\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\treturn result.Exists(), nil\n}\n\n// Undelete restores the deleted file at `path`.\nfunc (cl *Client) Undelete(path string) error {\n\tcall := cl.api.Undelete(cl.ctx, func(p capnp.FS_undelete_Params) error {\n\t\treturn p.SetPath(path)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// DeletedNodes returns a list of deleted nodes under `root`.\nfunc (cl *Client) DeletedNodes(root string) ([]StatInfo, error) {\n\tcall := cl.api.DeletedNodes(cl.ctx, func(p capnp.FS_deletedNodes_Params) error {\n\t\treturn p.SetRoot(root)\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapNodes, err := result.Nodes()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tresults := []StatInfo{}\n\tfor idx := 0; idx < capNodes.Len(); idx++ {\n\t\tcapInfo := capNodes.At(idx)\n\t\tinfo, err := convertCapStatInfo(&capInfo)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tresults = append(results, *info)\n\t}\n\n\treturn results, err\n}\n\n// IsCached checks if file or directory at `path` is cached.\nfunc (cl *Client) IsCached(path string) (bool, error) {\n\tcall := cl.api.IsCached(cl.ctx, func(p capnp.FS_isCached_Params) error {\n\t\treturn p.SetPath(path)\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\treturn result.IsCached(), nil\n}\n\n// RecodeStream takes the stream at `path` and, if it is a file, re-encodes\n// the stream with the current settings retrieved from the hint system.\nfunc (cl *Client) RecodeStream(path string) error {\n\tcall := cl.api.RecodeStream(cl.ctx, func(p capnp.FS_recodeStream_Params) error {\n\t\treturn p.SetPath(path)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n" }, { "path": "client/fs_test.go", "content": "package client_test\n\nimport (\n\t\"bytes\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"sort\"\n\t\"testing\"\n\n\t\"github.com/sahib/brig/client\"\n\t\"github.com/sahib/brig/client/clienttest\"\n\t\"github.com/sahib/brig/repo/hints\"\n\tcolorLog \"github.com/sahib/brig/util/log\"\n\t\"github.com/sahib/brig/util/testutil\"\n\tlog \"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc init() {\n\tlog.SetLevel(log.WarnLevel)\n\tlog.SetFormatter(&colorLog.FancyLogFormatter{\n\t\tUseColors: true,\n\t})\n}\n\nfunc stringify(err error) string {\n\tif err == nil {\n\t\treturn \"\"\n\t}\n\n\treturn err.Error()\n}\n\nfunc withDaemon(t *testing.T, name string, fn func(ctl *client.Client)) {\n\trequire.NoError(\n\t\tt,\n\t\tclienttest.WithDaemon(name, func(ctl *client.Client) error {\n\t\t\tfn(ctl)\n\t\t\treturn nil\n\t\t}),\n\t)\n}\n\nfunc withDaemonPair(t *testing.T, nameA, nameB string, fn func(ctlA, ctlB *client.Client)) {\n\trequire.NoError(\n\t\tt,\n\t\tclienttest.WithDaemonPair(nameA, nameB, func(ctlA, ctlB *client.Client) error {\n\t\t\tfn(ctlA, ctlB)\n\t\t\treturn nil\n\t\t}),\n\t)\n}\n\nfunc TestStageAndCat(t *testing.T) {\n\twithDaemon(t, \"ali\", func(ctl *client.Client) {\n\t\tfd, err := ioutil.TempFile(\"\", \"brig-dummy-data\")\n\t\tpath := fd.Name()\n\t\tdefer os.RemoveAll(path)\n\n\t\texpected := testutil.CreateDummyBuf(2 * 1024 * 1024)\n\t\trequire.Nil(t, err, stringify(err))\n\t\t_, err = fd.Write(expected)\n\t\trequire.Nil(t, err, stringify(err))\n\t\trequire.Nil(t, fd.Close())\n\n\t\trequire.Nil(t, ctl.Stage(path, \"/hello\"))\n\t\trw, err := ctl.Cat(\"hello\", false)\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\tdata, err := ioutil.ReadAll(rw)\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\trequire.Equal(t, expected, data)\n\t\trequire.Nil(t, rw.Close())\n\t})\n}\n\nfunc TestStageAndCatStream(t *testing.T) {\n\twithDaemon(t, \"ali\", func(ctl *client.Client) {\n\t\tconst fileSize = 4 * 1024 * 1024\n\t\tr := io.LimitReader(&testutil.TenReader{}, fileSize)\n\t\terr := ctl.StageFromReader(\"/hello\", r)\n\t\trequire.NoError(t, err)\n\n\t\t// time.Sleep(time.Second)\n\t\trw, err := ctl.Cat(\"/hello\", false)\n\t\trequire.NoError(t, err)\n\n\t\tn, err := io.Copy(&testutil.TenWriter{}, rw)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, int64(fileSize), n)\n\t\trequire.NoError(t, rw.Close())\n\t})\n}\n\nfunc TestMkdir(t *testing.T) {\n\twithDaemon(t, \"ali\", func(ctl *client.Client) {\n\t\t// Create something nested with -p...\n\t\trequire.Nil(t, ctl.Mkdir(\"/a/b/c\", true))\n\n\t\t// Create it twice...\n\t\trequire.Nil(t, ctl.Mkdir(\"/a/b/c\", true))\n\n\t\t// Create something nested without -p\n\t\terr := ctl.Mkdir(\"/x/y/z\", false)\n\t\trequire.Contains(t, err.Error(), \"No such file\")\n\n\t\trequire.Nil(t, ctl.Mkdir(\"/x\", false))\n\t\trequire.Nil(t, ctl.Mkdir(\"/x/y\", false))\n\t\trequire.Nil(t, ctl.Mkdir(\"/x/y/z\", false))\n\n\t\tlst, err := ctl.List(\"/\", -1)\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\tpaths := []string{}\n\t\tfor _, info := range lst {\n\t\t\tpaths = append(paths, info.Path)\n\t\t}\n\n\t\tsort.Strings(paths)\n\t\trequire.Equal(t, paths, []string{\n\t\t\t\"/\",\n\t\t\t\"/a\",\n\t\t\t\"/a/b\",\n\t\t\t\"/a/b/c\",\n\t\t\t\"/x\",\n\t\t\t\"/x/y\",\n\t\t\t\"/x/y/z\",\n\t\t})\n\t})\n}\n\nfunc TestSyncBasic(t *testing.T) {\n\twithDaemonPair(t, \"ali\", \"bob\", func(aliCtl, bobCtl *client.Client) {\n\t\terr := aliCtl.StageFromReader(\"/ali_file\", bytes.NewReader([]byte{42}))\n\t\trequire.NoError(t, err)\n\n\t\terr = bobCtl.StageFromReader(\"/bob_file\", bytes.NewReader([]byte{23}))\n\t\trequire.NoError(t, err)\n\n\t\t_, err = aliCtl.Sync(\"bob\", true)\n\t\trequire.NoError(t, err)\n\n\t\t_, err = bobCtl.Sync(\"ali\", true)\n\t\trequire.NoError(t, err)\n\n\t\t// We cannot query the file contents, since the mock backend\n\t\t// does not yet store the file content anywhere.\n\t\tbobFileStat, err := aliCtl.Stat(\"/bob_file\")\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, \"/bob_file\", bobFileStat.Path)\n\n\t\taliFileStat, err := bobCtl.Stat(\"/ali_file\")\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, \"/ali_file\", aliFileStat.Path)\n\t})\n}\n\nfunc pathsFromListing(l []client.StatInfo) []string {\n\tresult := []string{}\n\tfor _, entry := range l {\n\t\tresult = append(result, entry.Path)\n\t}\n\n\treturn result\n}\n\nfunc TestSyncConflict(t *testing.T) {\n\twithDaemonPair(t, \"ali\", \"bob\", func(aliCtl, bobCtl *client.Client) {\n\t\t// Create two files with the same content on both sides:\n\t\terr := aliCtl.StageFromReader(\"/README\", bytes.NewReader([]byte{42}))\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\terr = bobCtl.StageFromReader(\"/README\", bytes.NewReader([]byte{42}))\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\t// Sync and check if the files are still equal:\n\t\t_, err = bobCtl.Sync(\"ali\", true)\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\taliFileStat, err := aliCtl.Stat(\"/README\")\n\t\trequire.Nil(t, err, stringify(err))\n\t\tbobFileStat, err := bobCtl.Stat(\"/README\")\n\t\trequire.Nil(t, err, stringify(err))\n\t\trequire.Equal(t, aliFileStat.ContentHash, bobFileStat.ContentHash)\n\n\t\t// Modify bob's side only. A sync should have no effect.\n\t\terr = bobCtl.StageFromReader(\"/README\", bytes.NewReader([]byte{43}))\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\t_, err = bobCtl.Sync(\"ali\", true)\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\tbobFileStat, err = bobCtl.Stat(\"/README\")\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\trequire.NotEqual(t, aliFileStat.ContentHash, bobFileStat.ContentHash)\n\n\t\t// Modify ali's side additionally. Now we should get a conflicting file.\n\t\terr = aliCtl.StageFromReader(\"/README\", bytes.NewReader([]byte{41}))\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\tdirs, err := bobCtl.List(\"/\", -1)\n\t\trequire.Nil(t, err, stringify(err))\n\t\trequire.Equal(t, []string{\"/\", \"/README\"}, pathsFromListing(dirs))\n\n\t\t_, err = bobCtl.Sync(\"ali\", true)\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\tdirs, err = bobCtl.List(\"/\", -1)\n\t\trequire.Nil(t, err, stringify(err))\n\t\trequire.Equal(\n\t\t\tt,\n\t\t\t[]string{\"/\", \"/README\", \"/README.conflict.0\"},\n\t\t\tpathsFromListing(dirs),\n\t\t)\n\t})\n}\n\nfunc TestSyncSeveralTimes(t *testing.T) {\n\twithDaemonPair(t, \"ali\", \"bob\", func(aliCtl, bobCtl *client.Client) {\n\t\terr := aliCtl.StageFromReader(\"/ali_file_1\", bytes.NewReader([]byte{1}))\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\t_, err = bobCtl.Sync(\"ali\", true)\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\tdirs, err := bobCtl.List(\"/\", -1)\n\t\trequire.Nil(t, err, stringify(err))\n\t\trequire.Equal(\n\t\t\tt,\n\t\t\t[]string{\"/\", \"/ali_file_1\"},\n\t\t\tpathsFromListing(dirs),\n\t\t)\n\n\t\terr = aliCtl.StageFromReader(\"/ali_file_2\", bytes.NewReader([]byte{2}))\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\t_, err = bobCtl.Sync(\"ali\", true)\n\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\tdirs, err = bobCtl.List(\"/\", -1)\n\t\trequire.Nil(t, err, stringify(err))\n\t\trequire.Equal(\n\t\t\tt,\n\t\t\t[]string{\"/\", \"/ali_file_1\", \"/ali_file_2\"},\n\t\t\tpathsFromListing(dirs),\n\t\t)\n\n\t\terr = aliCtl.StageFromReader(\"/ali_file_3\", bytes.NewReader([]byte{3}))\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\t_, err = bobCtl.Sync(\"ali\", true)\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\tdirs, err = bobCtl.List(\"/\", -1)\n\t\trequire.Nil(t, err, stringify(err))\n\t\trequire.Equal(\n\t\t\tt,\n\t\t\t[]string{\"/\", \"/ali_file_1\", \"/ali_file_2\", \"/ali_file_3\"},\n\t\t\tpathsFromListing(dirs),\n\t\t)\n\t})\n}\n\nfunc TestSyncPartial(t *testing.T) {\n\twithDaemonPair(t, \"ali\", \"bob\", func(aliCtl, bobCtl *client.Client) {\n\t\taliWhoami, err := aliCtl.Whoami()\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\tbobWhoami, err := bobCtl.Whoami()\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\trequire.Nil(t, aliCtl.RemoteSave([]client.Remote{\n\t\t\t{\n\t\t\t\tName: \"bob\",\n\t\t\t\tFingerprint: bobWhoami.Fingerprint,\n\t\t\t\tFolders: []client.RemoteFolder{\n\t\t\t\t\t{\n\t\t\t\t\t\tFolder: \"/photos\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t}))\n\n\t\trequire.Nil(t, bobCtl.RemoteSave([]client.Remote{\n\t\t\t{\n\t\t\t\tName: \"ali\",\n\t\t\t\tFingerprint: aliWhoami.Fingerprint,\n\t\t\t\tFolders: []client.RemoteFolder{\n\t\t\t\t\t{\n\t\t\t\t\t\tFolder: \"/photos\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t}))\n\n\t\terr = aliCtl.StageFromReader(\"/docs/ali_secret.txt\", bytes.NewReader([]byte{0}))\n\t\trequire.Nil(t, err, stringify(err))\n\t\terr = aliCtl.StageFromReader(\"/photos/ali.png\", bytes.NewReader([]byte{42}))\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\terr = bobCtl.StageFromReader(\"/docs/bob_secret.txt\", bytes.NewReader([]byte{0}))\n\t\trequire.Nil(t, err, stringify(err))\n\t\terr = bobCtl.StageFromReader(\"/photos/bob.png\", bytes.NewReader([]byte{23}))\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\t_, err = aliCtl.Sync(\"bob\", true)\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\t_, err = bobCtl.Sync(\"ali\", true)\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\t// We cannot query the file contents, since the mock backend\n\t\t// does not yet store the file content anywhere.\n\t\taliLs, err := aliCtl.List(\"/\", -1)\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\taliPaths := []string{}\n\t\tfor _, entry := range aliLs {\n\t\t\taliPaths = append(aliPaths, entry.Path)\n\t\t}\n\n\t\tbobLs, err := bobCtl.List(\"/\", -1)\n\t\trequire.Nil(t, err, stringify(err))\n\n\t\tbobPaths := []string{}\n\t\tfor _, entry := range bobLs {\n\t\t\tbobPaths = append(bobPaths, entry.Path)\n\t\t}\n\n\t\trequire.Equal(\n\t\t\tt,\n\t\t\t[]string{\n\t\t\t\t\"/\",\n\t\t\t\t\"/docs\",\n\t\t\t\t\"/photos\",\n\t\t\t\t\"/docs/ali_secret.txt\",\n\t\t\t\t\"/photos/ali.png\",\n\t\t\t\t\"/photos/bob.png\",\n\t\t\t},\n\t\t\taliPaths,\n\t\t)\n\n\t\trequire.Equal(\n\t\t\tt,\n\t\t\t[]string{\n\t\t\t\t\"/\",\n\t\t\t\t\"/docs\",\n\t\t\t\t\"/photos\",\n\t\t\t\t\"/docs/bob_secret.txt\",\n\t\t\t\t\"/photos/ali.png\",\n\t\t\t\t\"/photos/bob.png\",\n\t\t\t},\n\t\t\tbobPaths,\n\t\t)\n\t})\n}\n\nfunc TestSyncMovedFile(t *testing.T) {\n\twithDaemonPair(t, \"ali\", \"bob\", func(aliCtl, bobCtl *client.Client) {\n\t\trequire.NoError(t, aliCtl.StageFromReader(\"/ali-file\", bytes.NewReader([]byte{1, 2, 3})))\n\t\trequire.NoError(t, bobCtl.StageFromReader(\"/bob-file\", bytes.NewReader([]byte{4, 5, 6})))\n\n\t\taliDiff, err := aliCtl.Sync(\"bob\", true)\n\t\trequire.NoError(t, err)\n\n\t\tbobDiff, err := bobCtl.Sync(\"ali\", true)\n\t\trequire.NoError(t, err)\n\n\t\trequire.Equal(t, aliDiff.Added[0].Path, \"/bob-file\")\n\t\trequire.Equal(t, bobDiff.Added[0].Path, \"/ali-file\")\n\n\t\trequire.NoError(t, aliCtl.Move(\"/ali-file\", \"/bali-file\"))\n\n\t\tbobDiffAfter, err := bobCtl.Sync(\"ali\", true)\n\t\trequire.NoError(t, err)\n\n\t\trequire.Len(t, bobDiffAfter.Added, 0)\n\t\trequire.Len(t, bobDiffAfter.Removed, 0)\n\t\trequire.Len(t, bobDiffAfter.Moved, 1)\n\t})\n}\n\n// Regression test for:\n// https://github.com/sahib/brig/issues/56\nfunc TestSyncRemovedFile(t *testing.T) {\n\tlog.SetLevel(log.DebugLevel)\n\twithDaemonPair(t, \"ali\", \"bob\", func(aliCtl, bobCtl *client.Client) {\n\t\trequire.NoError(t, aliCtl.StageFromReader(\"/testfile\", bytes.NewReader([]byte{1, 2, 3})))\n\n\t\t// Bob should get the /testfile now.\n\t\tbobDiff, err := bobCtl.Sync(\"ali\", true)\n\t\trequire.NoError(t, err)\n\n\t\trequire.Equal(t, 1, len(bobDiff.Added))\n\t\trequire.Equal(t, bobDiff.Added[0].Path, \"/testfile\")\n\n\t\trequire.NoError(t, bobCtl.StageFromReader(\"/testfile\", bytes.NewReader([]byte{3, 2, 1})))\n\t\trequire.NoError(t, bobCtl.MakeCommit(\"bob changed testfile\"))\n\n\t\t// Remove the file at ali:\n\t\trequire.NoError(t, aliCtl.Remove(\"/testfile\"))\n\t\trequire.NoError(t, aliCtl.MakeCommit(\"removed testfile\"))\n\n\t\t// Sync and hope that we don't get the file back from bob:\n\t\taliDiff, err := aliCtl.Sync(\"bob\", true)\n\t\trequire.NoError(t, err)\n\n\t\t// Check if something was added.\n\t\trequire.Equal(t, 0, len(aliDiff.Added))\n\n\t\t// ...but also checked it's not marked as removed:\n\t\trequire.Equal(t, 0, len(aliDiff.Removed))\n\n\t\t_, err = aliCtl.Stat(\"/testfile\")\n\t\trequire.Error(t, err)\n\t})\n}\n\nfunc TestHints(t *testing.T) {\n\twithDaemon(t, \"ali\", func(ctl *client.Client) {\n\t\t// Add hint for directory.\n\n\t\tpath := \"/public/cat-meme.png\"\n\t\texpected := testutil.CreateDummyBuf(1024 * 1024)\n\t\trequire.NoError(t, ctl.Mkdir(\"/public\", true))\n\t\trequire.NoError(t, ctl.StageFromReader(path, bytes.NewReader(expected)))\n\n\t\tinfo, err := ctl.Stat(path)\n\t\trequire.NoError(t, err)\n\n\t\tdefHints := hints.Default()\n\t\trequire.Equal(t, string(defHints.CompressionAlgo), info.Hint.CompressionAlgo)\n\t\trequire.Equal(t, string(defHints.EncryptionAlgo), info.Hint.EncryptionAlgo)\n\t\trequire.Equal(t, false, info.IsRaw)\n\n\t\tnone := \"none\"\n\t\trequire.NoError(t, ctl.HintSet(\"/public\", &none, &none))\n\n\t\tinfo, err = ctl.Stat(path)\n\t\trequire.NoError(t, err)\n\n\t\trequire.Equal(t, \"none\", info.Hint.CompressionAlgo)\n\t\trequire.Equal(t, \"none\", info.Hint.EncryptionAlgo)\n\t\trequire.Equal(t, false, info.IsRaw)\n\n\t\trequire.NoError(t, ctl.RecodeStream(\"/public\"))\n\n\t\tinfo, err = ctl.Stat(path)\n\t\trequire.NoError(t, err)\n\n\t\trequire.Equal(t, \"none\", info.Hint.CompressionAlgo)\n\t\trequire.Equal(t, \"none\", info.Hint.EncryptionAlgo)\n\t\trequire.Equal(t, true, info.IsRaw)\n\n\t\t// Make sure it did not scramble the data:\n\t\tstream, err := ctl.Cat(path, true)\n\t\trequire.NoError(t, err)\n\t\tgot, err := ioutil.ReadAll(stream)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, expected, got)\n\t})\n}\n" }, { "path": "client/net_cmds.go", "content": "package client\n\nimport (\n\t\"errors\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/sahib/brig/server/capnp\"\n\tcapnplib \"zombiezen.com/go/capnproto2\"\n)\n\n////////////////////////\n// REMOTE LIST ACCESS //\n////////////////////////\n\n// RemoteFolder is a single folder shared with a remote.\ntype RemoteFolder struct {\n\tFolder string `yaml:\"Folder\"`\n\tReadOnly bool `yaml:\"ReadOnly\"`\n\tConflictStrategy string `yaml:\"ConflictStrategy\"`\n}\n\n// Remote describes a single remote in the remote list.\ntype Remote struct {\n\tName string `yaml:\"Name\"`\n\tFingerprint string `yaml:\"Fingerprint\"`\n\tFolders []RemoteFolder `yaml:\"Folders,flow\"`\n\tAutoUpdate bool `yaml:\"AutoUpdate\"`\n\tConflictStrategy string `yaml:\"ConflictStrategy\"`\n\tAcceptPush bool `yaml:\"AcceptPush\"`\n}\n\nfunc capRemoteToRemote(capRemote capnp.Remote) (*Remote, error) {\n\tremoteName, err := capRemote.Name()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tremoteFp, err := capRemote.Fingerprint()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tremoteFolders, err := capRemote.Folders()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tconflictStrategy, err := capRemote.ConflictStrategy()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfolders := []RemoteFolder{}\n\tfor idx := 0; idx < remoteFolders.Len(); idx++ {\n\t\tfolder := remoteFolders.At(idx)\n\t\tfolderName, err := folder.Folder()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tcs, err := folder.ConflictStrategy()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tfolders = append(folders, RemoteFolder{\n\t\t\tFolder: folderName,\n\t\t\tReadOnly: folder.ReadOnly(),\n\t\t\tConflictStrategy: cs,\n\t\t})\n\t}\n\n\treturn &Remote{\n\t\tName: remoteName,\n\t\tFingerprint: remoteFp,\n\t\tFolders: folders,\n\t\tAutoUpdate: capRemote.AcceptAutoUpdates(),\n\t\tAcceptPush: capRemote.AcceptPush(),\n\t\tConflictStrategy: conflictStrategy,\n\t}, nil\n}\n\nfunc remoteToCapRemote(remote Remote, seg *capnplib.Segment) (*capnp.Remote, error) {\n\tcapRemote, err := capnp.NewRemote(seg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := capRemote.SetName(remote.Name); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := capRemote.SetFingerprint(string(remote.Fingerprint)); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := capRemote.SetConflictStrategy(remote.ConflictStrategy); err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapFolders, err := capnp.NewRemoteFolder_List(seg, int32(len(remote.Folders)))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor idx, folder := range remote.Folders {\n\t\tcapFolder, err := capnp.NewRemoteFolder(seg)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tcapFolder.SetReadOnly(folder.ReadOnly)\n\t\tif err := capFolder.SetFolder(folder.Folder); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif err := capFolder.SetConflictStrategy(folder.ConflictStrategy); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif err := capFolders.Set(idx, capFolder); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tif err := capRemote.SetFolders(capFolders); err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapRemote.SetAcceptAutoUpdates(remote.AutoUpdate)\n\tcapRemote.SetAcceptPush(remote.AcceptPush)\n\treturn &capRemote, nil\n}\n\n// RemoteAddOrUpdate adds a new remote described in `remote`.\n// We thus authenticate this remote.\nfunc (cl *Client) RemoteAddOrUpdate(remote Remote) error {\n\tcall := cl.api.RemoteAddOrUpdate(cl.ctx, func(p capnp.Net_remoteAddOrUpdate_Params) error {\n\t\tcapRemote, err := remoteToCapRemote(remote, p.Segment())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn p.SetRemote(*capRemote)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// RemoteByName adds a new remote described in `remote`.\n// We thus authenticate this remote.\nfunc (cl *Client) RemoteByName(name string) (Remote, error) {\n\tcall := cl.api.RemoteByName(cl.ctx, func(p capnp.Net_remoteByName_Params) error {\n\t\treturn p.SetName(name)\n\t})\n\n\tres, err := call.Struct()\n\tif err != nil {\n\t\treturn Remote{}, err\n\t}\n\n\tcapRmt, err := res.Remote()\n\tif err != nil {\n\t\treturn Remote{}, err\n\t}\n\n\trmt, err := capRemoteToRemote(capRmt)\n\tif err != nil {\n\t\treturn Remote{}, err\n\t}\n\n\treturn *rmt, nil\n}\n\n// RemoteUpdate Updates the contents of `remote`.\nfunc (cl *Client) RemoteUpdate(remote Remote) error {\n\tcall := cl.api.RemoteUpdate(cl.ctx, func(p capnp.Net_remoteUpdate_Params) error {\n\t\tcapRemote, err := remoteToCapRemote(remote, p.Segment())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn p.SetRemote(*capRemote)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// RemoteRm removes a remote by `name` from the remote list.\nfunc (cl *Client) RemoteRm(name string) error {\n\tcall := cl.api.RemoteRm(cl.ctx, func(p capnp.Net_remoteRm_Params) error {\n\t\treturn p.SetName(name)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// RemoteClear clears all of the remote list.\nfunc (cl *Client) RemoteClear() error {\n\tcall := cl.api.RemoteClear(cl.ctx, func(p capnp.Net_remoteClear_Params) error {\n\t\treturn nil\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// RemoteLs lists all remotes in the remote list.\nfunc (cl *Client) RemoteLs() ([]Remote, error) {\n\tcall := cl.api.RemoteLs(cl.ctx, func(p capnp.Net_remoteLs_Params) error {\n\t\treturn nil\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapRemotes, err := result.Remotes()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tremotes := []Remote{}\n\tfor idx := 0; idx < capRemotes.Len(); idx++ {\n\t\tcapRemote := capRemotes.At(idx)\n\t\tremote, err := capRemoteToRemote(capRemote)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tremotes = append(remotes, *remote)\n\t}\n\n\treturn remotes, nil\n}\n\n// RemoteSave swaps the contents of the remote lists with the contents of `remotes`.\nfunc (cl *Client) RemoteSave(remotes []Remote) error {\n\tcall := cl.api.RemoteSave(cl.ctx, func(p capnp.Net_remoteSave_Params) error {\n\t\tseg := p.Segment()\n\t\tcapRemotes, err := capnp.NewRemote_List(seg, int32(len(remotes)))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfor idx, remote := range remotes {\n\t\t\tcapRemote, err := remoteToCapRemote(remote, seg)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tif err := capRemotes.Set(idx, *capRemote); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn p.SetRemotes(capRemotes)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// LocateResult is a result returned by Locate()\ntype LocateResult struct {\n\tName string\n\tAddr string\n\tMask []string\n\tFingerprint string\n}\n\nfunc capLrToLr(capLr capnp.LocateResult) (*LocateResult, error) {\n\tname, err := capLr.Name()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\taddr, err := capLr.Addr()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tmask, err := capLr.Mask()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfingerprint, err := capLr.Fingerprint()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &LocateResult{\n\t\tAddr: addr,\n\t\tName: name,\n\t\tMask: strings.Split(mask, \",\"),\n\t\tFingerprint: fingerprint,\n\t}, nil\n}\n\n// NetLocate tries to find other remotes by searching of `who` described by `mask`.\n// It will at max. take `timeoutSec` to search. This operation might take some time.\n// The return channel will yield a LocateResult once a new result is available.\nfunc (cl *Client) NetLocate(who, mask string, timeoutSec float64) (chan *LocateResult, error) {\n\tcall := cl.api.NetLocate(cl.ctx, func(p capnp.Net_netLocate_Params) error {\n\t\tp.SetTimeoutSec(float64(timeoutSec))\n\n\t\tif err := p.SetLocateMask(mask); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn p.SetWho(who)\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tticket := result.Ticket()\n\tresultCh := make(chan *LocateResult)\n\n\tgo func() {\n\t\tdefer close(resultCh)\n\n\t\tfor {\n\t\t\tnextCall := cl.api.NetLocateNext(cl.ctx, func(p capnp.Net_netLocateNext_Params) error {\n\t\t\t\tp.SetTicket(ticket)\n\t\t\t\treturn nil\n\t\t\t})\n\n\t\t\tresult, err := nextCall.Struct()\n\t\t\tif err != nil {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tif !result.HasResult() {\n\t\t\t\tbreak\n\t\t\t}\n\n\t\t\tcapLr, err := result.Result()\n\t\t\tif err != nil {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tlr, err := capLrToLr(capLr)\n\t\t\tif err != nil {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tresultCh <- lr\n\t\t}\n\t}()\n\n\treturn resultCh, nil\n}\n\n// RemotePing pings a remote by the name `who`.\nfunc (cl *Client) RemotePing(who string) (float64, error) {\n\tcall := cl.api.RemotePing(cl.ctx, func(p capnp.Net_remotePing_Params) error {\n\t\treturn p.SetWho(who)\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\treturn result.Roundtrip(), nil\n}\n\n// Whoami describes the current user state\ntype Whoami struct {\n\tCurrentUser string\n\tOwner string\n\tFingerprint string\n\tIsOnline bool\n}\n\n// Whoami describes our own identity.\nfunc (cl *Client) Whoami() (*Whoami, error) {\n\tcall := cl.api.Whoami(cl.ctx, func(p capnp.Net_whoami_Params) error {\n\t\treturn nil\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapWhoami, err := result.Whoami()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\twhoami := &Whoami{}\n\twhoami.CurrentUser, err = capWhoami.CurrentUser()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\twhoami.Fingerprint, err = capWhoami.Fingerprint()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\twhoami.Owner, err = capWhoami.Owner()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\twhoami.IsOnline = capWhoami.IsOnline()\n\treturn whoami, nil\n}\n\n// NetConnect connects to the ipfs network.\nfunc (cl *Client) NetConnect() error {\n\t_, err := cl.api.Connect(cl.ctx, func(p capnp.Net_connect_Params) error {\n\t\treturn nil\n\t}).Struct()\n\treturn err\n}\n\n// NetDisconnect disconnects from the ipfs network.\nfunc (cl *Client) NetDisconnect() error {\n\t_, err := cl.api.Disconnect(cl.ctx, func(p capnp.Net_disconnect_Params) error {\n\t\treturn nil\n\t}).Struct()\n\treturn err\n}\n\n// RemoteStatus is a entry in the remote online list.\n// Fingerprint is not necessarily filled.\ntype RemoteStatus struct {\n\tRemote Remote\n\tLastSeen time.Time\n\tRoundtrip time.Duration\n\tErr error\n\tAuthenticated bool\n}\n\nfunc capRemoteStatusToRemoteStatus(capStatus capnp.RemoteStatus) (*RemoteStatus, error) {\n\tcapRemote, err := capStatus.Remote()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tremote, err := capRemoteToRemote(capRemote)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tmsg, err := capStatus.Error()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tlastSeenStamp, err := capStatus.LastSeen()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tlastSeen := time.Now()\n\tif lastSeenStamp != \"\" {\n\t\tlastSeen, err = time.Parse(time.RFC3339, lastSeenStamp)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tpingErr := errors.New(msg)\n\tif len(msg) == 0 {\n\t\tpingErr = nil\n\t}\n\n\troundtripMs := time.Duration(capStatus.RoundtripMs()) * time.Millisecond\n\treturn &RemoteStatus{\n\t\tRemote: *remote,\n\t\tLastSeen: lastSeen,\n\t\tRoundtrip: roundtripMs,\n\t\tErr: pingErr,\n\t\tAuthenticated: capStatus.Authenticated(),\n\t}, nil\n}\n\n// RemoteOnlineList is like RemoteList but also includes IsOnline and Authenticated\n// status.\nfunc (cl *Client) RemoteOnlineList() ([]RemoteStatus, error) {\n\tcall := cl.api.RemoteOnlineList(cl.ctx, func(p capnp.Net_remoteOnlineList_Params) error {\n\t\treturn nil\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapStatuses, err := result.Infos()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tstatuses := []RemoteStatus{}\n\tfor idx := 0; idx < capStatuses.Len(); idx++ {\n\t\tcapStatus := capStatuses.At(idx)\n\t\tstatus, err := capRemoteStatusToRemoteStatus(capStatus)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tstatuses = append(statuses, *status)\n\t}\n\n\treturn statuses, nil\n}\n\n// Push sets a push request to `remoteName`. If `dryRun` is true,\n// the push won't be send but we will still check if the push is allowed.\nfunc (cl *Client) Push(remoteName string, dryRun bool) error {\n\tcall := cl.api.Push(cl.ctx, func(p capnp.Net_push_Params) error {\n\t\tp.SetDryRun(dryRun)\n\t\treturn p.SetRemoteName(remoteName)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n" }, { "path": "client/net_test.go", "content": "package client_test\n\nimport (\n\t\"bytes\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/sahib/brig/client\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestPush(t *testing.T) {\n\twithDaemonPair(t, \"ali\", \"bob\", func(aliCtl, bobCtl *client.Client) {\n\t\trequire.Nil(t, aliCtl.StageFromReader(\"/ali-file\", bytes.NewReader([]byte{1, 2, 3})))\n\n\t\terr := aliCtl.Push(\"bob\", true)\n\t\trequire.True(t, strings.HasSuffix(err.Error(), \"remote does not allow it\"))\n\n\t\taliRmt, err := bobCtl.RemoteByName(\"ali\")\n\t\trequire.Nil(t, err)\n\t\taliRmt.AcceptPush = true\n\t\trequire.Nil(t, bobCtl.RemoteAddOrUpdate(aliRmt))\n\n\t\terr = aliCtl.Push(\"bob\", true)\n\t\trequire.Nil(t, err)\n\n\t\terr = aliCtl.Push(\"bob\", false)\n\t\trequire.Nil(t, err)\n\n\t\t// There is a possible race condition here:\n\t\t// ``brig push`` only triggers the sync, but\n\t\t// waits only until the network message was sent.\n\t\t// It might take a small amount of time till the other\n\t\t// side managed to do the sync.\n\t\ttime.Sleep(250 * time.Millisecond)\n\n\t\t// bob should have ali file without him syncing explicitly.\n\t\t_, err = bobCtl.Stat(\"/ali-file\")\n\t\trequire.Nil(t, err)\n\n\t})\n}\n" }, { "path": "client/repo_cmds.go", "content": "package client\n\nimport (\n\t\"sort\"\n\n\tgwdb \"github.com/sahib/brig/gateway/db\"\n\t\"github.com/sahib/brig/server/capnp\"\n\th \"github.com/sahib/brig/util/hashlib\"\n\tcapnplib \"zombiezen.com/go/capnproto2\"\n)\n\n// Quit sends a quit signal to brigd.\nfunc (ctl *Client) Quit() error {\n\tcall := ctl.api.Quit(ctl.ctx, func(p capnp.Repo_quit_Params) error {\n\t\treturn nil\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// Ping pings the daemon to see if it is responding.\nfunc (ctl *Client) Ping() error {\n\tcall := ctl.api.Ping(ctl.ctx, func(p capnp.Repo_ping_Params) error {\n\t\treturn nil\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t_, err = result.Reply()\n\treturn err\n}\n\n// MountOptions holds the possible option for a single mount.\ntype MountOptions struct {\n\tReadOnly bool\n\tRootPath string\n\tOffline bool\n}\n\nfunc mountOptionsToCapnp(opts MountOptions, seg *capnplib.Segment) (*capnp.MountOptions, error) {\n\tcapOpts, err := capnp.NewMountOptions(seg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapOpts.SetReadOnly(opts.ReadOnly)\n\tcapOpts.SetOffline(opts.Offline)\n\tif err := capOpts.SetRootPath(opts.RootPath); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &capOpts, nil\n}\n\n// Mount creates a new mount at `mountPath` with `opts`.\nfunc (ctl *Client) Mount(mountPath string, opts MountOptions) error {\n\tcall := ctl.api.Mount(ctl.ctx, func(p capnp.Repo_mount_Params) error {\n\t\tcapOpts, err := mountOptionsToCapnp(opts, p.Segment())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif err := p.SetOptions(*capOpts); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn p.SetMountPath(mountPath)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// Unmount kills a previously created mount at `mountPath`.\nfunc (ctl *Client) Unmount(mountPath string) error {\n\tcall := ctl.api.Unmount(ctl.ctx, func(p capnp.Repo_unmount_Params) error {\n\t\treturn p.SetMountPath(mountPath)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// ConfigGet returns the value at `key`.\nfunc (ctl *Client) ConfigGet(key string) (string, error) {\n\tcall := ctl.api.ConfigGet(ctl.ctx, func(p capnp.Repo_configGet_Params) error {\n\t\treturn p.SetKey(key)\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\treturn result.Value()\n}\n\n// ConfigSet sets the key at `key` to `value`\nfunc (ctl *Client) ConfigSet(key, value string) error {\n\tcall := ctl.api.ConfigSet(ctl.ctx, func(p capnp.Repo_configSet_Params) error {\n\t\tif err := p.SetValue(value); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn p.SetKey(key)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// ConfigEntry is a single entry of the config.\ntype ConfigEntry struct {\n\tKey string\n\tVal string\n\tDoc string\n\tDefault string\n\tNeedsRestart bool\n}\n\nfunc configEntryFromCapnp(capEntry capnp.ConfigEntry) (*ConfigEntry, error) {\n\tkey, err := capEntry.Key()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tval, err := capEntry.Val()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdoc, err := capEntry.Doc()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdef, err := capEntry.Default()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &ConfigEntry{\n\t\tDefault: def,\n\t\tKey: key,\n\t\tVal: val,\n\t\tDoc: doc,\n\t\tNeedsRestart: capEntry.NeedsRestart(),\n\t}, nil\n}\n\n// ConfigAll returns all config entries with details.\nfunc (ctl *Client) ConfigAll() ([]ConfigEntry, error) {\n\tcall := ctl.api.ConfigAll(ctl.ctx, func(p capnp.Repo_configAll_Params) error {\n\t\treturn nil\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapPairs, err := result.All()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tentries := []ConfigEntry{}\n\tfor idx := 0; idx < capPairs.Len(); idx++ {\n\t\tcapEntry := capPairs.At(idx)\n\t\tentry, err := configEntryFromCapnp(capEntry)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tentries = append(entries, *entry)\n\t}\n\n\treturn entries, nil\n}\n\n// ConfigDoc gets the documentation for a single config entry at `key`.\nfunc (ctl *Client) ConfigDoc(key string) (ConfigEntry, error) {\n\tcall := ctl.api.ConfigDoc(ctl.ctx, func(p capnp.Repo_configDoc_Params) error {\n\t\treturn p.SetKey(key)\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn ConfigEntry{}, err\n\t}\n\n\tcapEntry, err := result.Desc()\n\tif err != nil {\n\t\treturn ConfigEntry{}, err\n\t}\n\n\tentry, err := configEntryFromCapnp(capEntry)\n\tif err != nil {\n\t\treturn ConfigEntry{}, err\n\t}\n\n\treturn *entry, nil\n}\n\n// VersionInfo describes the version of the server.\ntype VersionInfo struct {\n\tServerSemVer string\n\tServerRev string\n\tBackendSemVer string\n\tBackendRev string\n}\n\n// Version returns version information about the server.\nfunc (ctl *Client) Version() (*VersionInfo, error) {\n\tcall := ctl.api.Version(ctl.ctx, func(p capnp.Repo_version_Params) error {\n\t\treturn nil\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapVersion, err := result.Version()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tversion := &VersionInfo{}\n\tversion.ServerSemVer, err = capVersion.ServerVersion()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tversion.ServerRev, err = capVersion.ServerRev()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tversion.BackendSemVer, err = capVersion.BackendVersion()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tversion.BackendRev, err = capVersion.BackendRev()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn version, nil\n}\n\n// FstabAdd adds a new mount named `mountName` at `mountPath` with `opts`.\n// The mount will only be created after calling FstabApply.\nfunc (ctl *Client) FstabAdd(mountName, mountPath string, opts MountOptions) error {\n\tcall := ctl.api.FstabAdd(ctl.ctx, func(p capnp.Repo_fstabAdd_Params) error {\n\t\tif err := p.SetMountName(mountName); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif err := p.SetMountPath(mountPath); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tcapOpts, err := mountOptionsToCapnp(opts, p.Segment())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn p.SetOptions(*capOpts)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// FstabRemove removes a named mount called `mountName`.\nfunc (ctl *Client) FstabRemove(mountName string) error {\n\tcall := ctl.api.FstabRemove(ctl.ctx, func(p capnp.Repo_fstabRemove_Params) error {\n\t\treturn p.SetMountName(mountName)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// FstabApply will apply any changes made the filesystem tab.\n// This won't do anything if nothing was changed in the mean time.\nfunc (ctl *Client) FstabApply() error {\n\tcall := ctl.api.FstabApply(ctl.ctx, func(p capnp.Repo_fstabApply_Params) error {\n\t\treturn nil\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// FstabUnmountAll will unmount all currently mounted fstab entries.\nfunc (ctl *Client) FstabUnmountAll() error {\n\tcall := ctl.api.FstabUnmountAll(ctl.ctx, func(p capnp.Repo_fstabUnmountAll_Params) error {\n\t\treturn nil\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// FsTabEntry describes a single entry in the filesystem tab\ntype FsTabEntry struct {\n\tName string\n\tPath string\n\tRoot string\n\tActive bool\n\tReadOnly bool\n\tOffline bool\n}\n\nfunc capMountToMount(capEntry capnp.FsTabEntry) (*FsTabEntry, error) {\n\tname, err := capEntry.Name()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\troot, err := capEntry.Root()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tpath, err := capEntry.Path()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &FsTabEntry{\n\t\tPath: path,\n\t\tName: name,\n\t\tRoot: root,\n\t\tActive: capEntry.Active(),\n\t\tReadOnly: capEntry.ReadOnly(),\n\t\tOffline: capEntry.Offline(),\n\t}, nil\n}\n\n// FsTabList lists all fs tab entries.\nfunc (ctl *Client) FsTabList() ([]FsTabEntry, error) {\n\tcall := ctl.api.FstabList(ctl.ctx, func(p capnp.Repo_fstabList_Params) error {\n\t\treturn nil\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapMounts, err := result.Mounts()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tmounts := []FsTabEntry{}\n\tfor idx := 0; idx < capMounts.Len(); idx++ {\n\t\tcapMount := capMounts.At(idx)\n\t\tmount, err := capMountToMount(capMount)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tmounts = append(mounts, *mount)\n\t}\n\n\treturn mounts, nil\n}\n\n// GarbageItem is a single path that was reaped by the garbage collector.\ntype GarbageItem struct {\n\tPath string\n\tOwner string\n\tContent h.Hash\n}\n\n// GarbageCollect calls the backend (IPSF) garbage collector and returns the collected items.\nfunc (ctl *Client) GarbageCollect(aggressive bool) ([]*GarbageItem, error) {\n\tcall := ctl.api.GarbageCollect(ctl.ctx, func(p capnp.FS_garbageCollect_Params) error {\n\t\tp.SetAggressive(aggressive)\n\t\treturn nil\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfreed := []*GarbageItem{}\n\n\tcapFreed, err := result.Freed()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor idx := 0; idx < capFreed.Len(); idx++ {\n\t\tcapGcItem := capFreed.At(idx)\n\t\tgcItem := &GarbageItem{}\n\n\t\tgcItem.Owner, err = capGcItem.Owner()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tgcItem.Path, err = capGcItem.Path()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tcontent, err := capGcItem.Content()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tgcItem.Content, err = h.Cast(content)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tfreed = append(freed, gcItem)\n\t}\n\n\treturn freed, nil\n}\n\n// Become changes the current users to one of the users in the remote list.\nfunc (ctl *Client) Become(who string) error {\n\tcall := ctl.api.Become(ctl.ctx, func(p capnp.Repo_become_Params) error {\n\t\treturn p.SetWho(who)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// GatewayUser is a user that has access to the gateway.\ntype GatewayUser struct {\n\tName string\n\tPasswordHash string\n\tSalt string\n\tFolders []string\n\tRights []string\n}\n\n// GatewayUserAdd adds a new user to the user database.\n// `folders` is a list of directories he may access. It might be empty,\n// in which case he can access everything (same as []string{\"/\"})\nfunc (ctl *Client) GatewayUserAdd(name, password string, folders, rights []string) error {\n\tcall := ctl.api.GatewayUserAdd(ctl.ctx, func(p capnp.Repo_gatewayUserAdd_Params) error {\n\t\tif err := p.SetName(name); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif err := p.SetPassword(password); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif err := p.SetPassword(password); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tseg := p.Segment()\n\t\tcapFolders, err := capnplib.NewTextList(seg, int32(len(folders)))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfor idx, folder := range folders {\n\t\t\tif err := capFolders.Set(idx, folder); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\tif err := p.SetFolders(capFolders); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tcapRights, err := capnplib.NewTextList(seg, int32(len(rights)))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfor idx, right := range rights {\n\t\t\tif err := capRights.Set(idx, right); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn p.SetRights(capRights)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// GatewayUserRemove removes an existing user and will error out\n// if the said user does not exist.\nfunc (ctl *Client) GatewayUserRemove(name string) error {\n\tcall := ctl.api.GatewayUserRm(ctl.ctx, func(p capnp.Repo_gatewayUserRm_Params) error {\n\t\treturn p.SetName(name)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// GatewayUserList lists all currently existing users.\nfunc (ctl *Client) GatewayUserList() ([]GatewayUser, error) {\n\tcall := ctl.api.GatewayUserList(ctl.ctx, func(p capnp.Repo_gatewayUserList_Params) error {\n\t\treturn nil\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapUsers, err := result.Users()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tusers := []GatewayUser{}\n\tfor idx := 0; idx < capUsers.Len(); idx++ {\n\t\tcapUser := capUsers.At(idx)\n\t\tgwuser, err := gwdb.UserFromCapnp(capUser)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tusers = append(users, GatewayUser{\n\t\t\tName: gwuser.Name,\n\t\t\tSalt: gwuser.Salt,\n\t\t\tPasswordHash: gwuser.PasswordHash,\n\t\t\tFolders: gwuser.Folders,\n\t\t\tRights: gwuser.Rights,\n\t\t})\n\t}\n\n\treturn users, err\n}\n\n// DebugProfilePort will get the port of pprof server in the backend.\n// The port changes during daemon restarts.\nfunc (ctl *Client) DebugProfilePort() (int, error) {\n\tcall := ctl.api.DebugProfilePort(ctl.ctx, func(p capnp.Repo_debugProfilePort_Params) error {\n\t\treturn nil\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn -1, err\n\t}\n\n\treturn int(result.Port()), nil\n}\n\n// Hint is a container for configuring streams.\ntype Hint struct {\n\t// Path is the path the hint applies to (recursively)\n\tPath string\n\n\t// CompressionAlgo can be an algorithm or \"guess\"\n\t// to let brig choose a suitable one.\n\tCompressionAlgo string\n\n\t// EncryptionAlgo must be a valid encryption algorithm.\n\tEncryptionAlgo string\n}\n\n// HintSet remembers the given settings at `path` (and below)\nfunc (ctl *Client) HintSet(path string, compressionAlgo, encryptionAlgo *string) error {\n\tcall := ctl.api.HintSet(ctl.ctx, func(p capnp.Repo_hintSet_Params) error {\n\t\tcapHint, err := capnp.NewHint(p.Segment())\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif err := capHint.SetPath(path); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif compressionAlgo != nil {\n\t\t\tif err := capHint.SetCompressionAlgo(*compressionAlgo); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\tif encryptionAlgo != nil {\n\t\t\tif err := capHint.SetEncryptionAlgo(*encryptionAlgo); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn p.SetHint(capHint)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// HintRemove removes the hint at `path`.\nfunc (ctl *Client) HintRemove(path string) error {\n\tcall := ctl.api.HintRemove(ctl.ctx, func(p capnp.Repo_hintRemove_Params) error {\n\t\treturn p.SetPath(path)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\nfunc convertCapHint(capHint capnp.Hint) (*Hint, error) {\n\tpath, err := capHint.Path()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcompressionAlgo, err := capHint.CompressionAlgo()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tencryptionAlgo, err := capHint.EncryptionAlgo()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &Hint{\n\t\tPath: path,\n\t\tEncryptionAlgo: encryptionAlgo,\n\t\tCompressionAlgo: compressionAlgo,\n\t}, nil\n}\n\n// HintList lists all hints that are currently set.\nfunc (ctl *Client) HintList() ([]Hint, error) {\n\tcall := ctl.api.HintList(ctl.ctx, func(p capnp.Repo_hintList_Params) error {\n\t\treturn nil\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapHints, err := result.Hints()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\thints := []Hint{}\n\n\tfor idx := 0; idx < capHints.Len(); idx++ {\n\t\thint, err := convertCapHint(capHints.At(idx))\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\thints = append(hints, *hint)\n\t}\n\n\t// Sort for display convenience:\n\tsort.Slice(hints, func(i, j int) bool {\n\t\treturn hints[i].Path < hints[j].Path\n\t})\n\n\treturn hints, nil\n}\n" }, { "path": "client/vcs_cmds.go", "content": "package client\n\nimport (\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/sahib/brig/server/capnp\"\n\th \"github.com/sahib/brig/util/hashlib\"\n)\n\n// MakeCommit creates a new commit from the current staging area.\n// The commit will have the message `msg`.\nfunc (ctl *Client) MakeCommit(msg string) error {\n\tcall := ctl.api.Commit(ctl.ctx, func(p capnp.VCS_commit_Params) error {\n\t\treturn p.SetMsg(msg)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// Commit describes a single commit in more detail.\ntype Commit struct {\n\tHash h.Hash\n\tMsg string\n\tTags []string\n\tDate time.Time\n}\n\nfunc convertCapCommit(capEntry *capnp.Commit) (*Commit, error) {\n\tresult := Commit{}\n\tmodTimeStr, err := capEntry.Date()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := result.Date.UnmarshalText([]byte(modTimeStr)); err != nil {\n\t\treturn nil, err\n\t}\n\n\tresult.Hash, err = capEntry.Hash()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tresult.Msg, err = capEntry.Msg()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\ttagList, err := capEntry.Tags()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\ttags := []string{}\n\tfor idx := 0; idx < tagList.Len(); idx++ {\n\t\ttag, err := tagList.At(idx)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\ttags = append(tags, tag)\n\t}\n\n\tresult.Tags = tags\n\treturn &result, nil\n}\n\n// Log lists all commits, starting with the newest one.\nfunc (ctl *Client) Log() ([]Commit, error) {\n\tcall := ctl.api.Log(ctl.ctx, func(p capnp.VCS_log_Params) error {\n\t\treturn nil\n\t})\n\n\tresults := []Commit{}\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tentries, err := result.Entries()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor idx := 0; idx < entries.Len(); idx++ {\n\t\tcapEntry := entries.At(idx)\n\t\tresult, err := convertCapCommit(&capEntry)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tresults = append(results, *result)\n\t}\n\n\treturn results, nil\n}\n\n// Tag tags a commit (`rev`) with a certain `name`.\nfunc (ctl *Client) Tag(rev, name string) error {\n\tcall := ctl.api.Tag(ctl.ctx, func(p capnp.VCS_tag_Params) error {\n\t\tif err := p.SetTagName(name); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn p.SetRev(rev)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// Untag removes the `name` tag.\nfunc (ctl *Client) Untag(name string) error {\n\tcall := ctl.api.Untag(ctl.ctx, func(p capnp.VCS_untag_Params) error {\n\t\treturn p.SetTagName(name)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// Reset restores the content of `path` to the state at `rev`.\n// If `force` is true, it will overwrite the staging area if it needs to.\nfunc (ctl *Client) Reset(path, rev string, force bool) error {\n\tcall := ctl.api.Reset(ctl.ctx, func(p capnp.VCS_reset_Params) error {\n\t\tif err := p.SetPath(path); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tp.SetForce(force)\n\t\treturn p.SetRev(rev)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// Change describes a change of a node between two commits.\ntype Change struct {\n\tPath string\n\tMask []string\n\tHead *Commit\n\tNext *Commit\n\n\tMovedTo string\n\tWasPreviouslyAt string\n\n\tIsPinned bool\n\tIsExplicit bool\n}\n\n// History returns a detailed set of changes that happened to the node at `path`.\nfunc (ctl *Client) History(path string) ([]*Change, error) {\n\tcall := ctl.api.History(ctl.ctx, func(p capnp.VCS_history_Params) error {\n\t\treturn p.SetPath(path)\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\thistList, err := result.History()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tresults := []*Change{}\n\tfor idx := 0; idx < histList.Len(); idx++ {\n\t\tentry := histList.At(idx)\n\t\tpath, err := entry.Path()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tchange, err := entry.Change()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tcapHeadCmt, err := entry.Head()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\thead, err := convertCapCommit(&capHeadCmt)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tmovedTo, err := entry.MovedTo()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\twasPreviouslyAt, err := entry.WasPreviouslyAt()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tvar next *Commit\n\t\tif entry.HasNext() {\n\t\t\tcapNextCmt, err := entry.Next()\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\n\t\t\tnext, err = convertCapCommit(&capNextCmt)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t}\n\n\t\tresults = append(results, &Change{\n\t\t\tPath: path,\n\t\t\tMask: strings.Split(change, \"|\"),\n\t\t\tHead: head,\n\t\t\tNext: next,\n\t\t\tMovedTo: movedTo,\n\t\t\tWasPreviouslyAt: wasPreviouslyAt,\n\t\t\tIsPinned: entry.IsPinned(),\n\t\t\tIsExplicit: entry.IsExplicit(),\n\t\t})\n\t}\n\n\treturn results, nil\n}\n\n// DiffPair is a pair of nodes that were changed in some way.\ntype DiffPair struct {\n\tSrc StatInfo\n\tDst StatInfo\n}\n\n// Diff gives a detailed overview over the changes between two commits.\ntype Diff struct {\n\tAdded []StatInfo\n\tRemoved []StatInfo\n\tIgnored []StatInfo\n\tMissing []StatInfo\n\n\tMoved []DiffPair\n\tMerged []DiffPair\n\tConflict []DiffPair\n}\n\n// IsEmpty reports if a diff is completely empty (i.e. nothing changed)\nfunc (df *Diff) IsEmpty() bool {\n\treturn 0 == 0+\n\t\tlen(df.Added)+\n\t\tlen(df.Removed)+\n\t\tlen(df.Ignored)+\n\t\tlen(df.Missing)+\n\t\tlen(df.Moved)+\n\t\tlen(df.Merged)+\n\t\tlen(df.Conflict)\n}\n\nfunc convertDiffList(lst capnp.StatInfo_List) ([]StatInfo, error) {\n\tinfos := []StatInfo{}\n\n\tfor idx := 0; idx < lst.Len(); idx++ {\n\t\tcapInfo := lst.At(idx)\n\t\tinfo, err := convertCapStatInfo(&capInfo)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tinfos = append(infos, *info)\n\t}\n\n\treturn infos, nil\n}\n\nfunc convertDiffPairList(lst capnp.DiffPair_List) ([]DiffPair, error) {\n\tpairs := []DiffPair{}\n\tfor idx := 0; idx < lst.Len(); idx++ {\n\t\tcapPair := lst.At(idx)\n\t\tcapSrc, err := capPair.Src()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tcapDst, err := capPair.Dst()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tsrcInfo, err := convertCapStatInfo(&capSrc)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tdstInfo, err := convertCapStatInfo(&capDst)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tpairs = append(pairs, DiffPair{\n\t\t\tSrc: *srcInfo,\n\t\t\tDst: *dstInfo,\n\t\t})\n\t}\n\n\treturn pairs, nil\n}\n\nfunc convertCapDiffToDiff(capDiff capnp.Diff) (*Diff, error) {\n\tdiff := &Diff{}\n\n\tlst, err := capDiff.Added()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdiff.Added, err = convertDiffList(lst)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tlst, err = capDiff.Missing()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdiff.Missing, err = convertDiffList(lst)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tlst, err = capDiff.Removed()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdiff.Removed, err = convertDiffList(lst)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tlst, err = capDiff.Ignored()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdiff.Ignored, err = convertDiffList(lst)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tpairs, err := capDiff.Moved()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdiff.Moved, err = convertDiffPairList(pairs)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tpairs, err = capDiff.Merged()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdiff.Merged, err = convertDiffPairList(pairs)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tpairs, err = capDiff.Conflict()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdiff.Conflict, err = convertDiffPairList(pairs)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn diff, nil\n}\n\n// MakeDiff creates a diff between the commits at `remoteRev` and `localRev`.\n// If `needFetch` is true, the data is first updated from the remote.\nfunc (ctl *Client) MakeDiff(local, remote, localRev, remoteRev string, needFetch bool) (*Diff, error) {\n\tcall := ctl.api.MakeDiff(ctl.ctx, func(p capnp.VCS_makeDiff_Params) error {\n\t\tif err := p.SetLocalOwner(local); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif err := p.SetRemoteOwner(remote); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif err := p.SetLocalRev(localRev); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tp.SetNeedFetch(needFetch)\n\t\treturn p.SetRemoteRev(remoteRev)\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapDiff, err := result.Diff()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn convertCapDiffToDiff(capDiff)\n}\n\n// Fetch updates our internal copy of the data of `remote`.\nfunc (ctl *Client) Fetch(remote string) error {\n\tcall := ctl.api.Fetch(ctl.ctx, func(p capnp.VCS_fetch_Params) error {\n\t\treturn p.SetWho(remote)\n\t})\n\n\t_, err := call.Struct()\n\treturn err\n}\n\n// Sync triggers a sync with the data from `remote`.\n// If `needFetch` is true, the data is first updated from the remote.\nfunc (ctl *Client) Sync(remote string, needFetch bool) (*Diff, error) {\n\tcall := ctl.api.Sync(ctl.ctx, func(p capnp.VCS_sync_Params) error {\n\t\tp.SetNeedFetch(needFetch)\n\t\treturn p.SetWithWhom(remote)\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapDiff, err := result.Diff()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn convertCapDiffToDiff(capDiff)\n}\n\n// CommitInfo is like a stat(2) for commits.\nfunc (ctl *Client) CommitInfo(rev string) (bool, *Commit, error) {\n\tcall := ctl.api.CommitInfo(ctl.ctx, func(p capnp.VCS_commitInfo_Params) error {\n\t\treturn p.SetRev(rev)\n\t})\n\n\tresult, err := call.Struct()\n\tif err != nil {\n\t\treturn false, nil, err\n\t}\n\n\tif !result.IsValidRef() {\n\t\treturn false, nil, nil\n\t}\n\n\tcapCmt, err := result.Commit()\n\tif err != nil {\n\t\treturn false, nil, err\n\t}\n\n\tcmt, err := convertCapCommit(&capCmt)\n\tif err != nil {\n\t\treturn false, nil, err\n\t}\n\n\treturn true, cmt, nil\n}\n" }, { "path": "cmd/bug.go", "content": "package cmd\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"net/url\"\n\t\"os\"\n\t\"os/exec\"\n\t\"strings\"\n\n\t\"github.com/fatih/color\"\n\t\"github.com/sahib/brig/client\"\n\t\"github.com/sahib/brig/version\"\n\t\"github.com/toqueteos/webbrowser\"\n\t\"github.com/urfave/cli\"\n)\n\nconst (\n\treportURL = \"https://github.com/sahib/brig/issues/new?\"\n)\n\n// printError simply prints a nicely formatted error to stderr.\nfunc printError(msg string) {\n\tfmt.Fprintln(os.Stderr, color.RedString(\"*** \")+msg)\n}\n\n// cmdOutput runs a command at `path` with `args` and returns it's output.\n// No real error checking is done, on errors an empty string is returned.\nfunc cmdOutput(path string, args ...string) string {\n\tout, err := exec.Command(path, args...).Output() // #nosec\n\tif err != nil {\n\t\t// No other error checking here, `brig bug` is best effort.\n\t\tprintError(fmt.Sprintf(\"failed to run %s %s\", path, strings.Join(args, \" \")))\n\t\treturn \"\"\n\t}\n\n\treturn strings.TrimSpace(string(out))\n}\n\n// handleBugReport compiles a report of useful info when providing a bug report.\nfunc handleBugReport(ctx *cli.Context) error {\n\tbuf := &bytes.Buffer{}\n\tfmt.Fprintln(buf, `Please answer these questions before submitting your issue.\nPlease include anything else you think is helpful. Thanks!\n\n### What did you do?\n\n### What did you expect to see?\n\n### What did you see instead?\n\n### Do you still see this issue with a development binary?\n\n### Did you check if a similar bug report was already opened?\n\n### System details:`)\n\n\tfmt.Fprintf(buf, \"go version: ``%s``\\n\", cmdOutput(\"go\", \"version\"))\n\tfmt.Fprintf(buf, \"uname -s -v -m: ``%s``\\n\", cmdOutput(\"uname\", \"-s\", \"-v\", \"-m\"))\n\tfmt.Fprintf(buf, \"IPFS config: ``%s``\\n\", cmdOutput(\"ipfs\", \"config\", \"show\"))\n\tfmt.Fprintf(buf, \"\\n\")\n\n\tfmt.Fprintf(\n\t\tbuf,\n\t\t\"brig client version: ``%s [build: %s]``\\n\",\n\t\tversion.String(),\n\t\tversion.BuildTime,\n\t)\n\n\tdaemonURL, err := guessDaemonURL(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tctl, err := client.Dial(context.Background(), daemonURL)\n\tif err == nil {\n\t\t// Try to get the server side / ipfs version.\n\t\tversion, err := ctl.Version()\n\t\tif err == nil {\n\t\t\tfmt.Fprintf(\n\t\t\t\tbuf,\n\t\t\t\t\"brig server version: ``%s+%s``\\n\",\n\t\t\t\tversion.ServerSemVer,\n\t\t\t\tversion.ServerRev,\n\t\t\t)\n\t\t\tfmt.Fprintf(\n\t\t\t\tbuf,\n\t\t\t\t\"IPFS Version: ``%s+%s``\\n\",\n\t\t\t\tversion.BackendSemVer,\n\t\t\t\tversion.BackendRev,\n\t\t\t)\n\t\t}\n\t} else {\n\t\tprintError(\"Cannot get server and IPFS version.\")\n\t\tprintError(\"If it is possible to start the daemon, do it now.\")\n\t\tprintError(\"This will make the bug report more helpful. Thanks.\")\n\t}\n\n\tprintToStdout := ctx.Bool(\"stdout\")\n\tif !printToStdout {\n\t\t// Try to open the issue tracker for convinience:\n\t\turlVal := url.Values{}\n\t\turlVal.Set(\"body\", buf.String())\n\t\tif err := webbrowser.Open(reportURL + urlVal.Encode()); err != nil {\n\t\t\tprintToStdout = true\n\t\t}\n\t}\n\n\tif printToStdout {\n\t\t// If not, ask the user to print it directly:\n\t\tif !ctx.Bool(\"stdout\") {\n\t\t\tprintError(\"I failed to open the issue tracker in your browser.\")\n\t\t\tprintError(\"Please paste the underlying text manually at this URL:\")\n\t\t\tprintError(\"https://github.com/sahib/brig/issues\")\n\t\t}\n\t\tfmt.Println(buf.String())\n\t}\n\n\treturn nil\n}\n" }, { "path": "cmd/debug.go", "content": "package cmd\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"os\"\n\n\t\"github.com/dustin/go-humanize\"\n\t\"github.com/mr-tron/base58\"\n\t\"github.com/sahib/brig/catfs/mio\"\n\t\"github.com/sahib/brig/client\"\n\t\"github.com/sahib/brig/fuse/fusetest\"\n\t\"github.com/sahib/brig/repo/hints\"\n\t\"github.com/sahib/brig/util/testutil\"\n\t\"github.com/urfave/cli\"\n)\n\nfunc handleDebugPprofPort(ctx *cli.Context, ctl *client.Client) error {\n\tport, err := ctl.DebugProfilePort()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif port > 0 {\n\t\tfmt.Println(port)\n\t} else {\n\t\tfmt.Println(\"Profiling is not enabled.\")\n\t\tfmt.Println(\"Enable daemon.enable_pprof and restart.\")\n\t}\n\n\treturn nil\n}\n\nfunc readDebugKey(ctx *cli.Context) ([]byte, error) {\n\tkeyB58 := ctx.String(\"key\")\n\tkey, err := base58.Decode(keyB58)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn key, nil\n}\n\nfunc handleDebugDecodeStream(ctx *cli.Context) error {\n\tkey, err := readDebugKey(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfd, err := ioutil.TempFile(\"\", \"\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdefer fd.Close()\n\tdefer os.Remove(fd.Name())\n\n\t_, err = io.Copy(fd, os.Stdin)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t_, err = fd.Seek(0, io.SeekStart)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tstream, err := mio.NewOutStream(fd, ctx.Bool(\"raw\"), key)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t_, err = io.Copy(os.Stdout, stream)\n\treturn err\n}\n\nfunc handleDebugEncodeStream(ctx *cli.Context) error {\n\tkey, err := readDebugKey(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\thint := hints.Hint{\n\t\tEncryptionAlgo: hints.EncryptionHint(ctx.String(\"encryption\")),\n\t\tCompressionAlgo: hints.CompressionHint(ctx.String(\"compression\")),\n\t}\n\n\tif !hint.IsValid() {\n\t\treturn fmt.Errorf(\"invalid encryption or compression\")\n\t}\n\n\tr, _, err := mio.NewInStream(os.Stdin, \"\", key, hint)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t_, err = io.Copy(os.Stdout, r)\n\treturn err\n}\n\nfunc readStreamSized(ctx *cli.Context) (uint64, error) {\n\treturn humanize.ParseBytes(ctx.String(\"size\"))\n}\n\nfunc handleDebugTenSource(ctx *cli.Context) error {\n\ts, err := readStreamSized(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\ttr := &testutil.TenReader{}\n\t_, err = io.Copy(os.Stdout, io.LimitReader(tr, int64(s)))\n\treturn err\n}\n\nfunc handleDebugTenSink(ctx *cli.Context) error {\n\ts, err := readStreamSized(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\ttw := &testutil.TenWriter{}\n\tn, err := io.Copy(tw, os.Stdin)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif int64(s) != n {\n\t\treturn fmt.Errorf(\"expected %d, got %d bytes\", s, n)\n\t}\n\n\treturn nil\n}\n\nfunc handleDebugFuseMock(ctx *cli.Context) error {\n\topts := fusetest.Options{\n\t\tCatfsPath: ctx.String(\"catfs-path\"),\n\t\tMountPath: ctx.String(\"mount-path\"),\n\t\tIpfsPathOrMultiaddr: ctx.String(\"ipfs-path-or-multiaddr\"),\n\t\tURL: ctx.String(\"url\"),\n\t\tMountReadOnly: ctx.Bool(\"mount-ro\"),\n\t\tMountOffline: ctx.Bool(\"mount-offline\"),\n\t}\n\n\treturn fusetest.Launch(opts)\n}\n" }, { "path": "cmd/exit_codes.go", "content": "package cmd\n\nconst (\n\t// Success is the same as EXIT_SUCCESS in C\n\tSuccess = iota\n\n\t// BadArgs passed to cli; not our fault.\n\tBadArgs\n\n\t// BadPassword passed to prompt or switch; not our fault.\n\tBadPassword\n\n\t// DaemonNotResponding means the daemon does not respond in timely fashion.\n\t// Probably our fault.\n\tDaemonNotResponding\n\n\t// UnknownError is an uncategorized error, probably our fault.\n\tUnknownError\n)\n" }, { "path": "cmd/fs_handlers.go", "content": "package cmd\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"path\"\n\t\"path/filepath\"\n\t\"regexp\"\n\t\"strconv\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/sahib/brig/cmd/tabwriter\"\n\t\"github.com/sahib/brig/util\"\n\n\t\"github.com/dustin/go-humanize\"\n\t\"github.com/fatih/color\"\n\t\"github.com/sahib/brig/client\"\n\t\"github.com/urfave/cli\"\n\n\t\"github.com/vbauerster/mpb\"\n\t\"github.com/vbauerster/mpb/decor\"\n\tterminal \"github.com/wayneashleyberry/terminal-dimensions\"\n)\n\nfunc handleStage(ctx *cli.Context, ctl *client.Client) error {\n\tlocalPath := ctx.Args().Get(0)\n\treadFromStdin := ctx.Bool(\"stdin\")\n\trepoPath := filepath.Base(localPath)\n\n\tif len(ctx.Args()) > 1 {\n\t\trepoPath = ctx.Args().Get(1)\n\t\tif localPath == \"-\" {\n\t\t\treadFromStdin = true\n\t\t}\n\t}\n\n\tif readFromStdin {\n\t\trepoPath = ctx.Args().Get(0)\n\t\treturn ctl.StageFromReader(repoPath, os.Stdin)\n\t}\n\n\tabsLocalPath, err := filepath.Abs(localPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"Failed to retrieve absolute path: %v\", err)\n\t}\n\n\t_, err = os.Stat(absLocalPath)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn handleStageDirectory(ctx, ctl, absLocalPath, repoPath)\n}\n\n// holds brig repoPaths twins (by content) to a OS file system localPath\ntype twins struct {\n\tlocalPath string\n\trepoPaths []string\n}\n\ntype walkOptions struct {\n\tdereference bool\n\tcontinueOnError bool\n}\n\nfunc walk(root, repoRoot string, depth int, opt walkOptions) (map[string]twins, error) {\n\t// toBeStaged map: key is local path, value is array of repoPaths using the local path\n\ttoBeStaged := make(map[string]twins)\n\tdepth++\n\tif depth > 255 {\n\t\treturn toBeStaged, fmt.Errorf(\"Exceeded allowed dereferencing depth for %v\", root)\n\t}\n\troot = filepath.Clean(root)\n\trepoRoot = filepath.Clean(repoRoot)\n\terr := filepath.Walk(root, func(childPath string, info os.FileInfo, err error) error {\n\t\trepoPath := filepath.Join(\"/\", repoRoot, childPath[len(root):])\n\n\t\tif opt.dereference && info.Mode()&os.ModeSymlink != 0 {\n\t\t\t// NOTE: `brig` does not have concept of symlink\n\t\t\t// The lack of native symlinks in `brig` has the following potential issues\n\t\t\t// * Ignoring cycles limits valid use cases.\n\t\t\t// * Not ignoring cycles opens room for malicious input.\n\t\t\t//\n\t\t\t// Here we implement a dereference of symlinks as a temporary measure\n\t\t\t// which works in the most likely user scenarios (we assume that user\n\t\t\t// is not malicious to herself and does not create infinite symlinked loops).\n\t\t\t// If the level of recursion or cycles\n\t\t\t// (where link points to itself directly or indirectly) is exceeded,\n\t\t\t// we just fail on such link.\n\t\t\t// Currently, we have a depth limit of 255 (see couple line above).\n\t\t\tresolvedPath, err := filepath.EvalSymlinks(childPath)\n\t\t\tif err != nil {\n\t\t\t\tmsg := fmt.Sprintf(\"Failed to resolve: %v: %v\", childPath, err)\n\t\t\t\tif opt.continueOnError {\n\t\t\t\t\tfmt.Fprintf(os.Stderr, \"WARNING: %s\\n\", msg)\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\t\t\t\treturn fmt.Errorf(msg)\n\t\t\t}\n\t\t\tinfo, err = os.Stat(resolvedPath)\n\t\t\tif err != nil {\n\t\t\t\tmsg := fmt.Sprintf(\"Failed to do os.Stat(%v): %v\", resolvedPath, err)\n\t\t\t\tif opt.continueOnError {\n\t\t\t\t\tfmt.Fprintf(os.Stderr, \"WARNING: %s\\n\", msg)\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\t\t\t\treturn fmt.Errorf(msg)\n\t\t\t}\n\t\t\tchildPath = resolvedPath\n\t\t\tif info.Mode().IsDir() {\n\t\t\t\textra, err := walk(childPath, repoPath, depth, opt)\n\t\t\t\tif err != nil {\n\t\t\t\t\tif opt.continueOnError {\n\t\t\t\t\t\tfmt.Fprintf(os.Stderr, \"WARNING: %s\\n\", err.Error())\n\t\t\t\t\t\treturn nil\n\t\t\t\t\t}\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tfor k, v := range extra {\n\t\t\t\t\tt, ok := toBeStaged[k]\n\t\t\t\t\tif !ok {\n\t\t\t\t\t\tt = twins{\n\t\t\t\t\t\t\tv.localPath,\n\t\t\t\t\t\t\t[]string{},\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t\tt.repoPaths = append(t.repoPaths, v.repoPaths...)\n\t\t\t\t\ttoBeStaged[k] = t\n\t\t\t\t}\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\n\t\tif info.Mode().IsRegular() {\n\t\t\tk, _ := inodeString(childPath)\n\t\t\tt, ok := toBeStaged[k]\n\t\t\tif !ok {\n\t\t\t\tt = twins{\n\t\t\t\t\tchildPath,\n\t\t\t\t\t[]string{},\n\t\t\t\t}\n\t\t\t}\n\t\t\tt.repoPaths = append(t.repoPaths, repoPath)\n\t\t\ttoBeStaged[k] = t\n\t\t}\n\n\t\treturn nil\n\t})\n\treturn toBeStaged, err\n}\n\nfunc makeParentDirIfNeeded(ctx *cli.Context, ctl *client.Client, path string) error {\n\tparent := filepath.Dir(path)\n\tinfo, err := ctl.Stat(parent)\n\tif err != nil {\n\t\tif yes, _ := regexp.MatchString(\"No such file or directory:\", err.Error()); yes {\n\t\t\tcreateParents := true\n\t\t\terr = ctl.Mkdir(parent, createParents)\n\t\t}\n\t\treturn err\n\t}\n\tif info.IsDir {\n\t\treturn nil\n\t}\n\treturn fmt.Errorf(\"Cannot make dir from existing non dir node %s\", parent)\n}\n\nfunc handleStageDirectory(ctx *cli.Context, ctl *client.Client, root, repoRoot string) error {\n\t// Links will be reflinked in the `man cp` sense,\n\t// i.e. resolved repoPaths will point to the same content and backend hash\n\n\troot = filepath.Clean(root)\n\trepoRoot = filepath.Clean(repoRoot)\n\n\topt := walkOptions{\n\t\tdereference: !ctx.Bool(\"no-dereference\"),\n\t\tcontinueOnError: ctx.Bool(\"continue-on-error\"),\n\t}\n\n\ttoBeStaged, err := walk(root, repoRoot, 0, opt)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to walk dir: %v: %v\", root, err)\n\t}\n\n\tif len(toBeStaged) == 0 {\n\t\t// This might happen if ask to stage a symlink pointing to a dir\n\t\t// but Walk does not travel symlinks and we end up with empty list.\n\t\treturn nil\n\t}\n\n\twidth, err := terminal.Width()\n\tif err != nil {\n\t\tfmt.Fprintf(os.Stderr, \"warning: failed to get terminal size: %s\\n\", err)\n\t\twidth = 80\n\t}\n\n\tpbars := mpb.New(\n\t\t// override default (80) width\n\t\tmpb.WithWidth(int(width)),\n\t\t// override default 120ms refresh rate\n\t\tmpb.WithRefreshRate(250*time.Millisecond),\n\t)\n\n\tname := \"ETA\"\n\tbar := pbars.AddBar(\n\t\tint64(len(toBeStaged)),\n\t\tmpb.PrependDecorators(\n\t\t\t// display our name with one space on the right\n\t\t\tdecor.Name(name, decor.WC{W: len(name) + 1, C: decor.DidentRight}),\n\t\t\t// replace ETA decorator with \"done\" message, OnComplete event\n\t\t\tdecor.OnComplete(\n\t\t\t\t// ETA decorator with ewma age of 60, and width reservation of 4\n\t\t\t\tdecor.EwmaETA(decor.ET_STYLE_GO, 60, decor.WC{W: 4}), \"done\",\n\t\t\t),\n\t\t),\n\t\tmpb.AppendDecorators(decor.Percentage()),\n\t)\n\n\ttype stageList struct {\n\t\tlocal string\n\t\trepoList []string\n\t}\n\n\tnWorkers := 20\n\tstart := time.Now()\n\tjobs := make(chan twins, nWorkers)\n\n\t// Start a bunch of workers that will do the actual adding:\n\tfor idx := 0; idx < nWorkers; idx++ {\n\t\tgo func() {\n\t\t\tfor {\n\t\t\t\ttwinsSet, ok := <-jobs\n\t\t\t\tif !ok {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tfirstToStage := \"\"\n\t\t\t\tfor i, repoPath := range twinsSet.repoPaths {\n\t\t\t\t\tif i == 0 {\n\t\t\t\t\t\tfirstToStage = repoPath\n\t\t\t\t\t\t// First occurrence is staged.\n\t\t\t\t\t\t// Stage creates all needed parent directories.\n\t\t\t\t\t\tif err := ctl.Stage(twinsSet.localPath, repoPath); err != nil {\n\t\t\t\t\t\t\tfmt.Fprintf(os.Stderr, \"failed to stage '%s' as '%s': %v\\n\", twinsSet.localPath, repoPath, err)\n\t\t\t\t\t\t\tbreak\n\t\t\t\t\t\t}\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t\t// Copy does not create parent directories. We take care of it.\n\t\t\t\t\tif err := makeParentDirIfNeeded(ctx, ctl, repoPath); err != nil {\n\t\t\t\t\t\tfmt.Fprintf(os.Stderr, \"failed to make the parent dir for '%s': %v\\n\", repoPath, err)\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\t\t\t\t\tif err := ctl.Copy(firstToStage, repoPath); err != nil {\n\t\t\t\t\t\tfmt.Fprintf(os.Stderr, \"failed copy of '%s' to '%s': %v\\n\", firstToStage, repoPath, err)\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\t// Notify the bar. The op time is used for the ETA.\n\t\t\t\t// The time is measured by \"start\" is NOT the time used to\n\t\t\t\t// stage a single file. This would only work in a non-parallel\n\t\t\t\t// environment, because the ETA would assume that one file took\n\t\t\t\t// 2s, so 1000 files must take 2000s. Instead it measures the\n\t\t\t\t// time between two time recordings, which are in the ideal\n\t\t\t\t// case around 1/n_workers * time_to_stage but it measures the\n\t\t\t\t// actual amount of parallelism that we achieve.\n\t\t\t\tbar.IncrBy(1, time.Since(start))\n\t\t\t\tstart = time.Now()\n\t\t\t}\n\t\t}()\n\t}\n\n\t// Send the jobs onward:\n\tfor _, v := range toBeStaged {\n\t\tjobs <- v\n\t}\n\n\tclose(jobs)\n\tpbars.Wait()\n\treturn nil\n}\n\nfunc handleCat(ctx *cli.Context, ctl *client.Client) error {\n\tpath := \"/\"\n\tif len(ctx.Args()) >= 1 {\n\t\tpath = ctx.Args().First()\n\t}\n\n\tinfo, err := ctl.Stat(path)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdoOffline := ctx.Bool(\"offline\")\n\n\tvar stream io.ReadCloser\n\tif info.IsDir {\n\t\tstream, err = ctl.Tar(path, doOffline)\n\t} else if ctx.Bool(\"stream\") {\n\t\treturn ctl.CatOnClient(path, doOffline, os.Stdout)\n\t} else {\n\t\tstream, err = ctl.Cat(path, doOffline)\n\t}\n\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer util.Closer(stream)\n\n\tif _, err := io.Copy(os.Stdout, stream); err != nil {\n\t\treturn ExitCode{\n\t\t\tUnknownError,\n\t\t\tfmt.Sprintf(\"cat: %v\", err),\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc handleRm(ctx *cli.Context, ctl *client.Client) error {\n\tpath := ctx.Args().First()\n\n\tif err := ctl.Remove(path); err != nil {\n\t\treturn ExitCode{\n\t\t\tUnknownError,\n\t\t\tfmt.Sprintf(\"rm: %v\", err),\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc handleMv(ctx *cli.Context, ctl *client.Client) error {\n\tsrcPath := ctx.Args().Get(0)\n\tdstPath := ctx.Args().Get(1)\n\treturn ctl.Move(srcPath, dstPath)\n}\n\nfunc handleCp(ctx *cli.Context, ctl *client.Client) error {\n\tsrcPath := ctx.Args().Get(0)\n\tdstPath := ctx.Args().Get(1)\n\treturn ctl.Copy(srcPath, dstPath)\n}\n\nfunc colorForSize(size uint64) func(f string, a ...interface{}) string {\n\tswitch {\n\tcase size >= 1024 && size < 1024<<10:\n\t\treturn color.CyanString\n\tcase size >= 1024<<10 && size < 1024<<20:\n\t\treturn color.YellowString\n\tcase size >= 1024<<20 && size < 1024<<30:\n\t\treturn color.RedString\n\tcase size >= 1024<<30:\n\t\treturn color.MagentaString\n\tdefault:\n\t\treturn func(f string, a ...interface{}) string {\n\t\t\treturn f\n\t\t}\n\t}\n}\n\nfunc userPrefixMap(users []string) map[string]string {\n\tm := make(map[string]string)\n\tfor _, user := range users {\n\t\tm[user] = user\n\t}\n\n\ttryAbbrev := func(abbrev string) bool {\n\t\tfor _, short := range m {\n\t\t\tif short == abbrev {\n\t\t\t\treturn false\n\t\t\t}\n\t\t}\n\n\t\treturn true\n\t}\n\n\tfor name := range m {\n\t\tatIdx := strings.Index(name, \"@\")\n\t\tif atIdx != -1 && tryAbbrev(name[:atIdx]) {\n\t\t\tm[name] = name[:atIdx]\n\t\t\tcontinue\n\t\t}\n\n\t\tslashIdx := strings.Index(name, \"/\")\n\t\tif slashIdx != -1 && tryAbbrev(name[:slashIdx]) {\n\t\t\tm[name] = name[:slashIdx]\n\t\t\tcontinue\n\t\t}\n\t}\n\n\treturn m\n}\n\nfunc formatHint(hint client.Hint) string {\n\treturn fmt.Sprintf(\"enc:%s-zip:%s\", hint.EncryptionAlgo, hint.CompressionAlgo)\n}\n\nfunc handleList(ctx *cli.Context, ctl *client.Client) error {\n\tmaxDepth := ctx.Int(\"depth\")\n\tif ctx.Bool(\"recursive\") {\n\t\tmaxDepth = -1\n\t}\n\n\troot := \"/\"\n\tif ctx.Args().Present() {\n\t\troot = ctx.Args().First()\n\t}\n\n\tentries, err := ctl.List(root, maxDepth)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\ttabW := tabwriter.NewWriter(\n\t\tos.Stdout, 0, 0, 2, ' ',\n\t\ttabwriter.StripEscape,\n\t)\n\n\ttmpl, err := readFormatTemplate(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif tmpl != nil {\n\t\tfor _, entry := range entries {\n\t\t\tif err := tmpl.Execute(os.Stdout, entry); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t}\n\n\tusers := []string{}\n\tfor _, entry := range entries {\n\t\tusers = append(users, entry.User)\n\t}\n\n\tuserMap := userPrefixMap(users)\n\n\tif len(entries) != 0 {\n\t\tuserColumn := \"\"\n\t\tif len(userMap) > 1 {\n\t\t\tuserColumn = \"USER\\t\"\n\t\t}\n\n\t\tfmt.Fprintf(tabW, \"SIZE\\tBKEND\\tMODTIME\\t%sPATH\\tPIN\\tCACHED\\tHINT\\n\", userColumn)\n\t}\n\n\tfor _, entry := range entries {\n\t\tpinState := \" \" + pinStateToSymbol(entry.IsPinned, entry.IsExplicit)\n\n\t\tvar coloredPath string\n\t\tif entry.IsDir {\n\t\t\tcoloredPath = color.GreenString(entry.Path)\n\t\t} else {\n\t\t\tcoloredPath = color.WhiteString(entry.Path)\n\t\t}\n\n\t\tuserEntry := \"\"\n\t\tif len(userMap) > 1 {\n\t\t\tuserEntry = color.GreenString(userMap[entry.User]) + \"\\t\"\n\t\t}\n\n\t\tisCached, err := ctl.IsCached(entry.Path)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tcachedState := \" \" + pinStateToSymbol(isCached, false)\n\n\t\tfmt.Fprintf(\n\t\t\ttabW,\n\t\t\t\"%s\\t%s\\t%s\\t%s%s\\t%s\\t%s\\t%s\\n\",\n\t\t\tcolorForSize(entry.Size)(humanize.Bytes(entry.Size)),\n\t\t\tcolorForSize(entry.Size)(humanize.Bytes(uint64(entry.CachedSize))),\n\t\t\tentry.ModTime.Format(\"2006-01-02 15:04:05 MST\"),\n\t\t\tuserEntry,\n\t\t\tcoloredPath,\n\t\t\tpinState,\n\t\t\tcachedState,\n\t\t\tformatHint(entry.Hint),\n\t\t)\n\t}\n\n\treturn tabW.Flush()\n}\n\nfunc handleTree(ctx *cli.Context, ctl *client.Client) error {\n\troot := \"/\"\n\tif ctx.NArg() > 0 {\n\t\troot = ctx.Args().First()\n\t}\n\n\tentries, err := ctl.List(root, -1)\n\tif err != nil {\n\t\treturn ExitCode{\n\t\t\tUnknownError,\n\t\t\tfmt.Sprintf(\"tree: %v\", err),\n\t\t}\n\t}\n\n\tshowTree(entries, &treeCfg{\n\t\tshowPin: true,\n\t})\n\treturn nil\n}\n\nfunc handleMkdir(ctx *cli.Context, ctl *client.Client) error {\n\tpath := ctx.Args().First()\n\tcreateParents := ctx.Bool(\"parents\")\n\n\tif err := ctl.Mkdir(path, createParents); err != nil {\n\t\treturn ExitCode{UnknownError, fmt.Sprintf(\"mkdir: %v\", err)}\n\t}\n\n\treturn nil\n}\n\nfunc handleShow(ctx *cli.Context, ctl *client.Client) error {\n\tpath := ctx.Args().First()\n\tisValidRef, cmt, err := ctl.CommitInfo(path)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif isValidRef {\n\t\treturn handleShowCommit(ctx, ctl, cmt)\n\t}\n\n\treturn handleShowFileOrDir(ctx, ctl, path)\n}\n\nfunc handleShowCommit(ctx *cli.Context, ctl *client.Client, cmt *client.Commit) error {\n\ttabW := tabwriter.NewWriter(\n\t\tos.Stdout, 0, 0, 2, ' ',\n\t\ttabwriter.StripEscape,\n\t)\n\n\tprintPair := func(name string, val interface{}) {\n\t\tfmt.Fprintf(\n\t\t\ttabW,\n\t\t\t\"%s\\t%v\\t\\n\",\n\t\t\tcolor.WhiteString(name),\n\t\t\tval,\n\t\t)\n\t}\n\n\tprintPair(\"Path\", cmt.Hash)\n\tprintPair(\"Tags\", strings.Join(cmt.Tags, \", \"))\n\tprintPair(\"ModTime\", cmt.Date.Format(time.RFC3339))\n\tprintPair(\"Message\", cmt.Msg)\n\ttabW.Flush()\n\n\tself, err := ctl.Whoami()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdiff, err := ctl.MakeDiff(\n\t\tself.CurrentUser,\n\t\tself.CurrentUser,\n\t\tcmt.Hash.B58String()+\"^\",\n\t\tcmt.Hash.B58String(),\n\t\tfalse,\n\t)\n\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif !diff.IsEmpty() {\n\t\tfmt.Println()\n\t\tfmt.Println(\"Here's what changed in this commit:\")\n\t\tfmt.Println()\n\t\tprintDiffTree(diff, false)\n\t}\n\n\treturn nil\n}\n\nfunc handleShowFileOrDir(ctx *cli.Context, ctl *client.Client, path string) error {\n\tinfo, err := ctl.Stat(path)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\ttmpl, err := readFormatTemplate(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif tmpl != nil {\n\t\treturn tmpl.Execute(os.Stdout, info)\n\t}\n\n\tisCached, err := ctl.IsCached(path)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tpinState := yesify(info.IsPinned)\n\texplicitState := yesify(info.IsExplicit)\n\tcachedState := yesify(isCached)\n\n\tnodeType := \"file\"\n\tif info.IsDir {\n\t\tnodeType = \"directory\"\n\t}\n\n\ttabW := tabwriter.NewWriter(\n\t\tos.Stdout, 0, 0, 2, ' ',\n\t\ttabwriter.StripEscape,\n\t)\n\n\tprintPair := func(name string, val interface{}) {\n\t\tfmt.Fprintf(\n\t\t\ttabW,\n\t\t\t\"%s\\t%v\\t\\n\",\n\t\t\tcolor.WhiteString(name),\n\t\t\tval,\n\t\t)\n\t}\n\n\tprintPair(\"Path\", info.Path)\n\tprintPair(\"User\", info.User)\n\tprintPair(\"Type\", nodeType)\n\tprintPair(\"Size\", fmt.Sprintf(\"%s (%d bytes)\", humanize.Bytes(info.Size), info.Size))\n\tprintPair(\"Backend Size\", fmt.Sprintf(\"%s (%d bytes)\", humanize.Bytes(uint64(info.CachedSize)), info.CachedSize))\n\tprintPair(\"Inode\", strconv.FormatUint(info.Inode, 10))\n\tprintPair(\"Pinned\", pinState)\n\tprintPair(\"Explicit\", explicitState)\n\tprintPair(\"Cached\", cachedState)\n\tprintPair(\"IsRaw\", yesify(info.IsRaw))\n\tprintPair(\"ModTime\", info.ModTime.Format(time.RFC3339))\n\tprintPair(\"Tree Hash\", info.TreeHash.B58String())\n\tprintPair(\"Content Hash\", info.ContentHash.B58String())\n\tprintPair(\"Hint\", formatHint(info.Hint))\n\n\tif !info.IsDir {\n\t\tprintPair(\"Backend Hash\", info.BackendHash.B58String())\n\t} else {\n\t\tprintPair(\"Backend Hash\", \"-\")\n\t}\n\n\treturn tabW.Flush()\n}\n\nfunc handleEdit(ctx *cli.Context, ctl *client.Client) error {\n\trepoPath := ctx.Args().First()\n\n\texists, err := ctl.Exists(repoPath)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdata := []byte{}\n\tif exists {\n\t\tr, err := ctl.Cat(repoPath, false)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tdefer util.Closer(r)\n\n\t\tdata, err = ioutil.ReadAll(r)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\ttempPath, err := editToPath(data, path.Ext(repoPath))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdefer func() {\n\t\tif err := os.Remove(tempPath); err != nil {\n\t\t\tfmt.Printf(\"Failed to remove temp file: %v\\n\", err)\n\t\t}\n\t}()\n\n\treturn ctl.Stage(tempPath, repoPath)\n}\n\nfunc handleTouch(ctx *cli.Context, ctl *client.Client) error {\n\trepoPath := ctx.Args().First()\n\treturn ctl.Touch(repoPath)\n}\n\nfunc handleTrashList(ctx *cli.Context, ctl *client.Client) error {\n\troot := \"/\"\n\tif firstArg := ctx.Args().First(); firstArg != \"\" {\n\t\troot = firstArg\n\t}\n\n\tnodes, err := ctl.DeletedNodes(root)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor _, node := range nodes {\n\t\tfmt.Println(node.Path)\n\t}\n\n\treturn nil\n}\n\nfunc handleTrashRemove(ctx *cli.Context, ctl *client.Client) error {\n\treturn ctl.Undelete(ctx.Args().First())\n}\n" }, { "path": "cmd/help.go", "content": "package cmd\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\n\t\"github.com/sahib/brig/repo/hints\"\n\t\"github.com/toqueteos/webbrowser\"\n\t\"github.com/urfave/cli\"\n)\n\ntype helpEntry struct {\n\tUsage string\n\tArgsUsage string\n\tDescription string\n\tComplete cli.BashCompleteFunc\n\tFlags []cli.Flag\n}\n\nfunc die(msg string) {\n\t// be really pedantic when help is missing.\n\t// it is a developer mistake after all and should be catched early.\n\tpanic(msg)\n}\n\nfunc compressionHintsToBullets() string {\n\td := hints.Default()\n\ts := []string{}\n\n\tfor _, algo := range hints.CompressionHints() {\n\t\tsuffix := \"\"\n\t\tif d.CompressionAlgo == algo {\n\t\t\tsuffix = \" (default)\"\n\t\t}\n\n\t\ts = append(s, fmt.Sprintf(\" * %s%s\", algo, suffix))\n\t}\n\n\treturn strings.Join(s, \"\\n\")\n}\n\nfunc encryptionHintsToBullets() string {\n\td := hints.Default()\n\ts := []string{}\n\n\tfor _, algo := range hints.EncryptionHints() {\n\t\tsuffix := \"\"\n\t\tif d.EncryptionAlgo == algo {\n\t\t\tsuffix = \" (default)\"\n\t\t}\n\t\ts = append(s, fmt.Sprintf(\" * %s%s\", algo, suffix))\n\t}\n\n\treturn strings.Join(s, \"\\n\")\n}\n\nvar helpTexts = map[string]helpEntry{\n\t\"init\": {\n\t\tUsage: \"Initialize a new repository.\",\n\t\tArgsUsage: \"\",\n\t\tComplete: completeArgsUsage,\n\t\tFlags: []cli.Flag{\n\t\t\t// duplicate of global repo, because it is convenient to\n\t\t\t// write »brig init --repo blah«.\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"repo\",\n\t\t\t\tUsage: \"Path to the repository. Only has effect for new daemons.\",\n\t\t\t\tValue: \"\",\n\t\t\t\tEnvVar: \"BRIG_PATH\",\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"backend,b\",\n\t\t\t\tValue: \"httpipfs\",\n\t\t\t\tUsage: \"What data backend to use for the new repo. One of `mock`, `httpipfs`. This cannot be changed later!\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"empty,e\",\n\t\t\t\tUsage: \"Do not create an initial README and no initial commit.\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"no-logo,n\",\n\t\t\t\tUsage: \"Do not display the super pretty logo on init.\",\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"ipfs-path-or-multiaddr\",\n\t\t\t\tUsage: \"Specify a path to an ipfs repo (/tmp/ipfs-repo) or a multiaddr to a running ipfs daemon (/ip4/127.0.0.1/tcp/5002)\",\n\t\t\t\tValue: \"\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"no-ipfs-setup\",\n\t\t\t\tUsage: \"Do not try to install and setup IPFS.\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"no-ipfs-config\",\n\t\t\t\tUsage: \"Do no changes in the IPFS config that are necessary for brig. Use only when you know what you're doing.\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"no-ipfs-optimization,o\",\n\t\t\t\tUsage: \"Do no changes in the IPFS config that will improve the performance of brig, but are not necessary to work.\",\n\t\t\t},\n\t\t},\n\t\tDescription: `Initialize a new repository with a certain backend.\n\n If BRIG_PATH or --repo is set, the new repository will be created at this\n place. If nothing is specified, the repo is created at \"~/.brig\". If the\n directory is not empty, brig will warn you about it and abort.\n\n The user name can be specified as pretty much any string, but it is recommended\n to use the special format »user@domain.something/resource«. This is similar to\n XMPP IDs. Specifying a resource can help you use the same name for different\n computers and specifying a domain makes it possible to indicate groups. This\n is especially important for commands like »brig net locate« but is not used\n extensively by anything else yet.\n\nEXAMPLES:\n\n # Easiest way to create a repository at /tmp/brig\n $ brig init --repo /tmp/brig ali@wonderland.org/rabbithole\n\n`,\n\t},\n\t\"whoami\": {\n\t\tUsage: \"Print the own remote identity including IPFS id, fingerprint and user name.\",\n\t\tComplete: completeArgsUsage,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"fingerprint,f\",\n\t\t\t\tUsage: \"Only print the own fingerprint\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"name,n\",\n\t\t\t\tUsage: \"Only print the own name\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"addr,a\",\n\t\t\t\tUsage: \"Only print the IPFS id portion of the fingerprint\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"key,k\",\n\t\t\t\tUsage: \"Only print the key portion of the fingerprint\",\n\t\t\t},\n\t\t},\n\t\tDescription: `This command prints your name, fingerprint and what store\n you are looking at. When you initialized your repository, you chose\n the name and a fingerprint (two longer hash values) was created for you.\n\nEXAMPLES:\n\n # Show the fingerprint only:\n $ brig whoami -f\n QmUYz9dbqnYPyHCLUi7ghtiwFbdU93MQKFH4qg8iXHWcPV:W1q4vzbvLPUVwDUUXxjQfnuYJxq2CYqbeqXPSv7pUr5NcP\n`,\n\t},\n\t\"remote\": {\n\t\tUsage: \"Add, list, remove and edit remotes.\",\n\t\tComplete: completeSubcommands,\n\t\tDescription: `\n A remote is the data needed to contact other instances of brig in the web.\n In order to add a remote, you need their fingerprint (as shown by »brig\n whoami«). This fingerprint should be exchanged in prior over a secure side\n channel (a secure instant messenger for example). Once both sides added each\n other as remotes they are said to be »authenticated«.\n\n Each remote can be configured further by specifying folders they may access\n or special settings like auto-updating. See the individual commands for more\n information.\n\n Also see the »net locate« command for details about finding other remotes.\n\nEXAMPLES:\n\n # Show a diff for each remote:\n $ brig remote list --format '{{ .Name }}' | xargs -n 1 brig diff\n`,\n\t},\n\t\"remote.add\": {\n\t\tUsage: \"Add/Update a remote under a handy name with their fingerprint.\",\n\t\tArgsUsage: \" \",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: \"\",\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"auto-update,a\",\n\t\t\t\tUsage: \"Take automatic updates from this node.\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"accept-push,p\",\n\t\t\t\tUsage: \"Allow this remote to push to our state.\",\n\t\t\t},\n\t\t\tcli.StringSliceFlag{\n\t\t\t\tName: \"folder,f\",\n\t\t\t\tUsage: \"Configure the folders this remote may see. Can be given more than once. If the first letter of the folder is »-« it is added as read-only.\",\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"conflict-strategy,c\",\n\t\t\t\tUsage: \"Which conflict strategy to apply (either »marker«, »ignore« or »embrace«)\",\n\t\t\t\tValue: \"\",\n\t\t\t},\n\t\t},\n\t},\n\t\"remote.remove\": {\n\t\tUsage: \"Remove a remote by name.\",\n\t\tArgsUsage: \"\",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: \"Remove a remote by name.\",\n\t},\n\t\"remote.list\": {\n\t\tUsage: \"List all remotes and their online status\",\n\t\tComplete: completeArgsUsage,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"offline,o\",\n\t\t\t\tUsage: \"Do not query the online status\",\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"format,f\",\n\t\t\t\tUsage: \"Format the output according to a template\",\n\t\t\t},\n\t\t},\n\t\tDescription: `\n This goes over every entry in your remote list and prints by default\n the remote name, fingerprint, rountrip, last seen timestamp and settings.\n\n You can format the output by using »--format« with one the following attributes:\n\n * .Name\n * .Fingerprint\n * .Folders\n * .AutoUpdate\n\n The syntax of the template is borrowed from Go. You can read about the details here:\n https://golang.org/pkg/text/template\n\n Note that this command will try to peek the fingerprint of each node, even\n if we did not authenticate him yet. If you do not want this, you should use\n »--offline«.\n\nEXAMPLES:\n\n $ brig rmt ls -f '{{ .Name }}' # Show each remote name, line by line.\n`,\n\t},\n\t\"remote.clear\": {\n\t\tUsage: \"Clear the complete remote list.\",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: \"Note that you cannot undo this operation!\",\n\t},\n\t\"remote.ping\": {\n\t\tUsage: \"Ping a remote.\",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: `Ping a remote and check if we can reach them.\n\n There is a small difference to the »remote list« command. »ping« will only\n work if both sides authenticated each other and can thus be used as a test\n for this. Additionally, it shows the roundtrip time (i.e. the time the ping\n request took to travel).\n\nEXAMPLES:\n\n $ brig rmt ping\n`,\n\t},\n\t\"remote.edit\": {\n\t\tUsage: \"Edit the current list.\",\n\t\tComplete: completeArgsUsage,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"yml,y\",\n\t\t\t\tValue: \"\",\n\t\t\t\tUsage: \"Directly overwrite remote list with yml file\",\n\t\t\t},\n\t\t},\n\t\tDescription: `\n Edit the current list using $EDITOR as YAML file.\n It will be updated once you exit your editor.`,\n\t},\n\t\"remote.auto-update\": {\n\t\tUsage: \"Enable auto-updating for this remote\",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: `When enabled you will get updates shortly after this remote made it.\n\nEXAMPLES:\n\n # Enable auto-updating both for bob and charlie.\n $ brig remote auto-update enable bob charlie\n\n # or shorter to prevent you from RSI:\n brig rmt au e bob charlie\n`,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"no-initial-sync,n\",\n\t\t\t\tUsage: \"Do not sync initially when upon enabling.\",\n\t\t\t},\n\t\t},\n\t},\n\t\"remote.accept-push\": {\n\t\tUsage: \"Allow receiving push requests from this remote.\",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: `When enabled, other remotes can do »brig push « to us.\n When we receive a push request we will sync with this remote.\n\nEXAMPLES:\n\n # Allow bob and charlie to push to us:\n $ brig remote accept-push enable bob charlie\n\n # or shorter to prevent you from RSI:\n brig rmt ap e bob charlie\n`,\n\t},\n\t\"remote.conflict-strategy\": {\n\t\tUsage: \"Change what conflict resolution strategy is used on conflicts.\",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: `The conflict strategy defines how to act on sync conflicts.\n There are three different types:\n\n - marker: Create a conflict file with the remote's version. (default)\n - ignore: Ignore the remote version completely and keep our version.\n - embrace: Take the remote version and replace ours with it.\n\n See also »brig config doc fs.sync.conflict_strategy«.\n In case of an empty string, the config value above is used.\n\nEXAMPLES:\n\n # Allow bob and charlie to push to us:\n $ brig remote conflict-strategy embrace bob charlie\n\n # or shorter to prevent you from RSI:\n brig rmt cs embrace bob charlie\n`,\n\t},\n\t\"remote.folder\": {\n\t\tUsage: \"Configure what folders a remote is allowed to see.\",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: `\n By default every remote is allowed to see all of your folders.\n You might want to share only specific folders with certain remotes.\n By adding folders to this list, you're limiting the nodes other remotes can see.\n\n If you do not specify any subcommand, this is a shortcut for »brig rmt f ls«`,\n\t},\n\t\"remote.folder.add\": {\n\t\tUsage: \"Add a remote folder for a specific remote.\",\n\t\tComplete: completeArgsUsage,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"read-only,r\",\n\t\t\t\tUsage: \"Add the folder as read-only.\",\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"conflict-strategy,c\",\n\t\t\t\tUsage: \"What conflict strategy to use for this specific folder. Overwrites per-remote conflict strategy.\",\n\t\t\t\tValue: \"\",\n\t\t\t},\n\t\t},\n\t\tDescription: `If a folder is added as read-only, we do not accept changes when syncing from remotes.\n\nEXAMPLES:\n\n $ brig remote folder add bob /public --read-only\n`,\n\t},\n\t\"remote.folder.set\": {\n\t\tUsage: \"Update the settings of a remote folder.\",\n\t\tComplete: completeArgsUsage,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"read-only,r\",\n\t\t\t\tUsage: \"Add the folder as read-only.\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"read-write,w\",\n\t\t\t\tUsage: \"Add the folder as read and writeable.\",\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"conflict-strategy,c\",\n\t\t\t\tUsage: \"What conflict strategy to use for this specific folder. Overwrites per-remote conflict strategy.\",\n\t\t\t\tValue: \"\",\n\t\t\t},\n\t\t},\n\t\tDescription: `This works exactly like »add« but overwrites an existing folder.\n\nEXAMPLES:\n\n $ brig remote folder set bob /public --read-only\n`,\n\t},\n\t\"remote.folder.remove\": {\n\t\tUsage: \"Remove a folder from a specific remote. \",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: ``,\n\t},\n\t\"remote.folder.clear\": {\n\t\tUsage: \"Clear all folders from a specific remote.\",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: ``,\n\t},\n\t\"remote.folder.list\": {\n\t\tUsage: \"List all allowed folders for a specific remote.\",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: ``,\n\t},\n\t\"pin\": {\n\t\tUsage: \"Commands to handle the pin state.\",\n\t\tArgsUsage: \"\",\n\t\tComplete: completeBrigPath(true, true),\n\t\tDescription: `Pinning a file to keep it in local storage.\n\n When you retrieve a file from a remote machine, the file will be cached (or\n maybe only blocks of it) for some time on your machine. If the file is not pinned,\n it might be collected by the garbage collector on the next run. The garbage collector\n is currently not invoked automatically, but can be activated via »brig gc«.\n\n Note that you can also pin files that you do not have cached locally. The\n pin does not download a file automatically currently. Until we have a\n proper way to do this, you can use »brig cat > /dev/null«.\n\n This command contains the subcommand 'add', but for usability reasons, »brig\n pin add « is the same as »brig pin «.\n\n See also the »gc« command as counterpart of pinning.\n`,\n\t},\n\t\"pin.add\": {\n\t\tUsage: \"Pin a file or directory to local storage\",\n\t\tArgsUsage: \"\",\n\t\tComplete: completeBrigPath(true, true),\n\t\tDescription: `A node that is pinned to local storage will not be\n deleted by the garbage collector.`,\n\t},\n\t\"pin.remove\": {\n\t\tUsage: \"Remove a pin\",\n\t\tArgsUsage: \"\",\n\t\tComplete: completeBrigPath(true, true),\n\t\tDescription: `A node that is pinned to local storage will not be\n deleted by the garbage collector.`,\n\t},\n\t\"pin.repin\": {\n\t\tUsage: \"Recaculate pinning based on fs.repin.{quota,min_depth,max_depth}\",\n\t\tArgsUsage: \"[]\",\n\t\tComplete: completeBrigPath(true, true),\n\t\tDescription: `Trigger a repin calculation.\n\n This uses the following configuration variables:\n\n - fs.repin.quota: Max. amount of data to store in a repository.\n - fs.repin.min_depth: Keep this many versions definitely pinned. Trumps quota.\n - fs.repin.max_depth: Unpin versions beyond this depth definitely. Trumps quota.\n\n If repin detects files that need to be unpinned, then it will first unpin all files\n that are beyond the max depth setting. If this is not sufficient to stay under the quota,\n it will delete old versions, layer by layer starting with the biggest version first.\n\n If the optional root path was specified, the repin is only run in this part\n of the filesystem. This can be used to give the repin algorithm a hint where\n the space should be reclaimed.\n `,\n\t},\n\t\"net\": {\n\t\tUsage: \"Commands that change or query the network status.\",\n\t\tComplete: completeSubcommands,\n\t\tDescription: `Most of these subcommands are somewhat low-level and are not often used.`,\n\t},\n\t\"net.offline\": {\n\t\tUsage: \"Prevent any online usage.\",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: `\n\n The daemon will be running after going offline.\n After going offline, other peers will not be able to\n contact you any more and vice versa. The daemon keeps running in this\n time and you can do all offline operations.\n\n BUGS: This currently does not prevent other nodes to contact us.\n Shutdown the IPFS daemon to be sure for now.`,\n\t},\n\t\"net.online\": {\n\t\tUsage: \"Allow online usage.\",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: `\n\n Opposite of »brig net offline«. This is the default state whenever the daemon starts.`,\n\t},\n\t\"net.status\": {\n\t\tUsage: \"Check if you're connected to the global network.\",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: `This will either print the string »online« or »offline«.`,\n\t},\n\t\"net.locate\": {\n\t\tUsage: \"Try to locate a remote by their name or by a part of it.\",\n\t\tArgsUsage: \"\",\n\t\tComplete: completeArgsUsage,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"t,timeout\",\n\t\t\t\tValue: \"10s\",\n\t\t\t\tUsage: \"Wait at most seconds before bailing out\",\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"m,mask\",\n\t\t\t\tValue: \"exact,domain,user,email\",\n\t\t\t\tUsage: \"Indicate what part of the id you want to query for\",\n\t\t\t},\n\t\t},\n\t\tDescription: `brig is able to find the fingerprint of other users (that\n are online) by a part of their name. See the help of »brig init« to see\n out of what components the name is built of.\n\n Each found item shows the name, the fingerprint and what part of their name\n matched with your query. Sometimes other peers are offline and cannot send\n your their fingerprint. In this case the peer will still be shown, but as\n »offline«.\n\n IMPORTANT: Locating a remote DOES NOT replace proper authentication. It is\n relatively easy to fake a fingerprint or even to have two peers with the same\n name. Always authenticate your peer properly via a sidechannel (mail,\n telephone, in person). »locate« is supposed to be only a help of discovering\n other nodes.\n\n Note that this operation might take quite a few seconds. Specifying »--timeout« can help,\n but currently it still might take longer than the given timeout.`,\n\t},\n\t\"status\": {\n\t\tUsage: \"Show what has changed in the current commit.\",\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"tree,t\",\n\t\t\t\tUsage: \"View the status as a tree listing.\",\n\t\t\t},\n\t\t},\n\t\tDescription: `This a shortcut for »brig diff HEAD CURR«.\nSee the »diff« command for more information.`,\n\t},\n\t\"diff\": {\n\t\tUsage: \"Show what changed between two commits.\",\n\t\tArgsUsage: \"[] [ [ []]]]\",\n\t\tComplete: completeArgsUsage,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"list,l\",\n\t\t\t\tUsage: \"Output the diff as simple list (like status does by default)\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"offline,o\",\n\t\t\t\tUsage: \"Do no fetch operation before computing the diff.\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"self,s\",\n\t\t\t\tUsage: \"Assume self as owner of both sides and compare only commits.\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"missing,m\",\n\t\t\t\tUsage: \"Show missing files in diff output.\",\n\t\t\t},\n\t\t},\n\t\tDescription: `View what sync would do when being called on the specified points in history.\n\n Diff does not show what changed inside of the files, but shows how the files\n themselves changed compared to the remote. To describe this, brig knows\n seven different change types:\n\n - Added (+): The file was added on the remote side.\n - Removed (-): The file was removed on the remote side.\n - Missing (_): The file is missing on the remote side (e.g. we added it)\n - Moved (→): The file was moved to a new location.\n - Ignored (*): This file was ignored because we chose to due to our settings.\n - Mergeable (⇄): Both sides have changes, but they can be merged.\n - Conflict (⚡): Both sides have changes but they conflict.\n\n Before computing the diff, it will try to fetch the metadata from the peer,\n if necessary. If you do not want this behaviour, use the »--offline« flag.\n\n See »brig commit« for a general explanation of commits.\n\nEXAMPLES:\n\n $ brig diff # Show diff from our CURR to our HEAD\n $ brig diff alice # Show diff from our CURR to alice's last state\n $ brig diff alice some_tag # Show diff from our CURR to 'some_tag' of alice\n $ brig diff alice bob HEAD HEAD # Show diff between alice and bob's HEAD\n $ brig diff -s HEAD CURR # Show diff between HEAD and CURR of alice\n`,\n\t},\n\t\"tag\": {\n\t\tUsage: \"Tag a commit with a specific name\",\n\t\tComplete: completeArgsUsage,\n\t\tArgsUsage: \" \",\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"delete,d\",\n\t\t\t\tUsage: \"Delete the tag instead of creating it\",\n\t\t\t},\n\t\t},\n\t\tDescription: `Give a name to a commit, which is easier to remember than the hash.\n You can use the name you gave in all places where brig requires you to specify a commit.\n\n There are three special tags pre-defined for you:\n\n - CURR: A reference to the staging commit.\n - HEAD: The last fully completed commit.\n - INIT: The very first commit in the chain.\n\n Tags are case insensitive. That means that »HEAD« and »head« mean the same.\n If you want to specify a commit by its index, you can use the special syntax\n »commit[$idx]« where »$idx« can be a zero-indexed number. The first commit\n has the index of zero.\n\n If you want to access the previous commit, you can also use the special\n syntax »$rev^« where »$rev« is any revision (either a commit hash, a tag\n name or anything else). The circumflex can be used more than once to go\n back further.\n\nEXAMPLES:\n\n $ brig tag SEfXUAH6AR my-tag-name # Name the commit SEfXUAH6AR 'my-tag-name'.\n $ brig tag -d my-tag-name # Delete the tag name again.\n $ brig tag HEAD^ previous-head # Tag the commit before the current HEAD with \"previous-head\".\n $ brig tag 'commit[1]' second # Tag the commit directly after init with \"second\".\n`,\n\t},\n\t\"log\": {\n\t\tUsage: \"Show all commits in a certain range\",\n\t\tComplete: completeArgsUsage,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"format,f\",\n\t\t\t\tUsage: \"Format the output according to a template\",\n\t\t\t},\n\t\t},\n\t\tDescription: `Show a list of commits from a start (--from) up to and end (--to).\n If omitted »--from INIT --to CURR« will be assumed.\n\n The output will show one commit per line, each including the (short) hash of the commit,\n the date it was committed and the (optional) commit message.\n`,\n\t},\n\t\"fetch\": {\n\t\tUsage: \"Fetch all metadata from another peer.\",\n\t\tArgsUsage: \"\",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: `This is a plumbing commands and most likely is only needed for debugging.\n\n Get all the latest metadata of a certain peer.\n This does not download any actual data, but only the metadata of it.\n You have to be authenticated to the user to get his data.\n\n Fetch will be done automatically by »sync« and »diff« and is usually\n only helpful when doing it together with »become«.`,\n\t},\n\t\"sync\": {\n\t\tUsage: \"Sync with another peer\",\n\t\tArgsUsage: \"\",\n\t\tComplete: completeArgsUsage,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"no-fetch,n\",\n\t\t\t\tUsage: \"Do not do a fetch before syncing.\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"quiet,q\",\n\t\t\t\tUsage: \"Do not print what changed.\",\n\t\t\t},\n\t\t},\n\t\tDescription: `Sync and merge all metadata of another peer with our metadata.\n After this operation you might see new files in your folder.\n Those files were not downloaded yet and will be only on the first access.\n\n It is recommended that your first check what will be synced with »brig diff«.\n\n When passing no arguments, 'sync' will synchronize with all online remotes.\n When passing a single argument, it will be used as the remote name to sync with.\n\n The symbols in the output prefixing every path have the following meaning:\n\n + The file is only present on the remote side.\n - The file was removed on the remote side.\n → The file was moved to a new location.\n * This file was ignored because we chose to, due to our settings.\n ⇄ Both sides have changes, but they are compatible and can be merged.\n ⚡ Both sides have changes, but they are incompatible and result in conflicts.\n _ The file is missing on the remote side.\n\n See also »brig help diff« for some more details.\n Files from other remotes are not pinned automatically.\n`,\n\t},\n\t\"push\": {\n\t\tUsage: \"Ask a remote to sync with us.\",\n\t\tComplete: completeArgsUsage,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"dry-run,d\",\n\t\t\t\tUsage: \"Do not the actual push, but check if we may push.\",\n\t\t\t},\n\t\t},\n\t\tDescription: ``,\n\t},\n\t\"commit\": {\n\t\tUsage: \"Create a new commit\",\n\t\tComplete: completeArgsUsage,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"message,m\",\n\t\t\t\tValue: \"\",\n\t\t\t\tUsage: \"Provide a meaningful commit message.\",\n\t\t\t},\n\t\t},\n\t\tDescription: `Create a new commit.\n\n The message (»--message«) is optional. If you do not pass it, a message will\n be generated which contains the current time. The commit history can be\n viewed by »brig log«.\n\n Think of commits as snapshots that can be created explicitly by you or even\n automated in an interval. It is important to remember that »commit« will\n only create a snapshot of the metadata. It is not guaranteed that you can\n still access the actual data of very old versions (See »brig help )\n\n You normally do not need to issue this command manually, since there is a\n loop inside of brig that will auto-commit every 5 minute (default; see the\n \"fs.autocommit.interval\" config key). Sync operations will also create\n commits implicitly and every change from the gateway side will also result\n in a commit.\n`,\n\t},\n\t\"reset\": {\n\t\tUsage: \"Reset a file or the whole commit to an old state.\",\n\t\tArgsUsage: \" []\",\n\t\tComplete: completeArgsUsage,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"force,f\",\n\t\t\t\tUsage: \"Reset even when there are changes in the staging area\",\n\t\t\t},\n\t\t},\n\t\tDescription: `Reset a file to an old state by specifying the commit it\n should be reverted to. If you do not pass »« the whole commit will be\n filled with the contents of the old commit.\n\n If you reset to an old commit and you have uncommitted changes, brig will warn you\n about that and refuse the »reset« unless you pass »--force«.\n\n Note for git users: It is not possible to go back in history and branch out\n from there. »reset« simply overwrites the staging commit (CURR) with an old\n state, thus keeping all the previous history. You can always jump back to\n the previous state. In other words: the reset operation of brig is not\n destructive. If you notice that you do not like the state you've reseted to,\n »brig reset head« will bring you back to the last known good state.\n`,\n\t},\n\t\"become\": {\n\t\tUsage: \"View the data of another user\",\n\t\tArgsUsage: \"\",\n\t\tComplete: completeArgsUsage,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"self,s\",\n\t\t\t\tUsage: \"Become self (i.e. the owner of the repository)\",\n\t\t\t},\n\t\t},\n\t\tDescription: `View the data of another user.\n\n This is a plumbing command and meant for debugging.\n\n You can temporarily explore the metadata of another user, by »becoming«\n them. Once you became a certain user (which needs to be in your remote list\n and on which you called »brig fetch« before), you can look around in the\n data like in yours. You can also modify files, but keep in mind that they\n will be reset on he next fetch.\n`,\n\t},\n\t\"history\": {\n\t\tUsage: \"Show the history of a file or directory\",\n\t\tArgsUsage: \"\",\n\t\tComplete: completeBrigPath(true, true),\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"empty,e\",\n\t\t\t\tUsage: \"Also show commits where nothing happens\",\n\t\t\t},\n\t\t},\n\t\tDescription: `Show a list of all changes that were made to this path.\n\n Not every change you ever made is recorded, but the change between each commit.\n In other words: If you modify a file, delete it and re-add all in one commit, then\n brig will see it only as one modification.\n\n Every line shows the type of change and what commits were involved. If it's a\n move, it will also show from and to where the path was moved.\n\n Possible types of changes are:\n\n - added: The file was added in this commit.\n - moved: The file was modified in this commit.\n - removed: The file was removed in this commit.\n - modified: The file was modified in this commit.\n\n Furthermore, the following combination are possible:\n\n - moved & modified: The file was moved and modified.\n - add & modified: The file was removed before and now re-added with different content.\n - moved & removed: The file was moved to another location.\n`,\n\t},\n\t\"stage\": {\n\t\tUsage: \"Add a local file to the storage.\",\n\t\tArgsUsage: \"( []|--stdin )\",\n\t\tComplete: completeLocalPath,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"stdin,i\",\n\t\t\t\tUsage: \"Read data from stdin.\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"no-dereference,P\",\n\t\t\t\tUsage: \"Never follow symbolic links.\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"continue-on-error,c\",\n\t\t\t\tUsage: \"Continue staging even if some parts fail to stage.\",\n\t\t\t},\n\t\t},\n\t\tDescription: `Read a local file (given by »local-path«) and try to read\n it. This is the conceptual equivalent of »git add«. The stream will be encrypted\n and possibly compressed before saving it to IPFS.\n\n If you omit »path«, the file will be added under the root\n directory, with the basename of »local-path«. You can change this by\n specifying where to save the local file by additionally passing »path«.\n\n Additionally you can read the file from standard input if you pass »--stdin«.\n In this case you pass only one path: The path where the stream is stored.\n\nEXAMPLES:\n\n $ brig stage file.png # gets added as /file.png\n $ brig stage file.png /photos/me.png # gets added as /photos/me.png\n $ cat file.png | brig --stdin /file.png # gets added as /file.png`,\n\t},\n\t\"touch\": {\n\t\tUsage: \"Create an empty file under the specified path\",\n\t\tArgsUsage: \"\",\n\t\tComplete: completeBrigPath(true, false),\n\t\tDescription: `Convenience command for adding empty files.\n\n If the file or directory already exists, the modification time is updated to\n the current timestamp (like the original touch(1) does).\n`,\n\t},\n\t\"cat\": {\n\t\tUsage: \"Output the content of a file to standard output\",\n\t\tArgsUsage: \"[]\",\n\t\tComplete: completeBrigPath(true, false),\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"offline,o\",\n\t\t\t\tUsage: \"Only output the file if it is cached locally.\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"stream,s\",\n\t\t\t\tUsage: \"Use experimental streaming implementation.\",\n\t\t\t},\n\t\t},\n\t\tDescription: `Decrypt and decompress the stream from IPFS and write it to standard output.\n\n When specifying a directory instead of a file, the directory content will be\n output as tar archive. This is useful when saving a whole directory tree to\n disk (see also EXAMPLES).\n\n When no path is specified, »/« is assumed and all contents are outputted as tar.\n\nEXAMPLES:\n\n # Output a single file:\n $ brig cat photo.png\n # Create a tar from root and unpack it to the current directory.\n $ brig cat | tar xfv -\n # Create .tar.gz out of of the /photos directory.\n $ brig cat photos | gzip -f > photos.tar.gz\n`,\n\t},\n\t\"show\": {\n\t\tUsage: \"Show metadata of a file or directory or commit\",\n\t\tArgsUsage: \"\",\n\t\tComplete: completeBrigPath(true, true),\n\t\tFlags: []cli.Flag{\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"format,f\",\n\t\t\t\tUsage: \"Format the output according to a template\",\n\t\t\t},\n\t\t},\n\t\tDescription: `Show all metadata attributes known for a file or directory.\n\n Path: Absolute path of the file inside of the storage.\n User: User which modified the file last.\n Type: »file« or »directory«.\n Size: Exact content size in bytes.\n Hash: Hash of the node.\n Inode: Internal inode. Also shown as inode in FUSE.\n IsPinned: »yes« if the file is pinned, »no« else.\n IsExplicit: »yes« if the file is pinned explicitly, »no« elsewise.\n IsRaw: »no« if the file was encoded by brig (using encryption or compression).\n ModTime: Timestamp of last modification.\n ContentHash: Content hash of the file before encryption.\n BackendHash: Hash of the node in ipfs (ipfs cat )\n TreeHash: Hash that is unique to this node.\n`,\n\t},\n\t\"rm\": {\n\t\tUsage: \"Remove a file or directory\",\n\t\tArgsUsage: \"\",\n\t\tComplete: completeBrigPath(true, true),\n\t\tDescription: `Remove a file or directory.\n\n In contrast to the usual rm(1) there is no --recursive switch.\n Directories are deleted recursively by default.\n\n Even after deleting files, you will be able to access its history by using\n the »brig history« command and bring them back via »brig reset«. If you want\n to restore a deleted entry you are able to with the »brig reset« command.\n`,\n\t},\n\t\"ls\": {\n\t\tUsage: \"List files and directories.\",\n\t\tArgsUsage: \"\",\n\t\tComplete: completeBrigPath(false, true),\n\t\tFlags: []cli.Flag{\n\t\t\tcli.IntFlag{\n\t\t\t\tName: \"depth,d\",\n\t\t\t\tUsage: \"Max depth to traverse\",\n\t\t\t\tValue: 1,\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"recursive,R\",\n\t\t\t\tUsage: \"Allow recursive traverse\",\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"format,f\",\n\t\t\t\tUsage: \"Format the output according to a template\",\n\t\t\t},\n\t\t},\n\t\tDescription: `List files an directories starting with »path«.\n If no »« is given, the root directory is assumed. Every line of »ls«\n shows a human readable size of each entry, the last modified time stamp, the\n user that last modified the entry (if there's more than one) and if the\n entry if pinned.\n`,\n\t},\n\t\"tree\": {\n\t\tUsage: \"List files and directories in a tree\",\n\t\tArgsUsage: \"\",\n\t\tComplete: completeBrigPath(false, true),\n\t\tFlags: []cli.Flag{\n\t\t\tcli.IntFlag{\n\t\t\t\tName: \"depth, d\",\n\t\t\t\tUsage: \"Max depth to traverse\",\n\t\t\t\tValue: -1,\n\t\t\t},\n\t\t},\n\t\tDescription: `Show entries in a tree(1)-like fashion.\n`,\n\t},\n\t\"mkdir\": {\n\t\tUsage: \"Create an empty directory\",\n\t\tArgsUsage: \"\",\n\t\tComplete: completeBrigPath(false, true),\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"parents, p\",\n\t\t\t\tUsage: \"Create parent directories as needed\",\n\t\t\t},\n\t\t},\n\t\tDescription: `Create an empty directory at the specified »path«.\n By default, parent directories are not created. You can use »--parents« to\n enable this behaviour.\n`,\n\t},\n\t\"mv\": {\n\t\tUsage: \"Move a file or directory from »src« to »dst«\",\n\t\tArgsUsage: \" \",\n\t\tComplete: completeBrigPath(true, true),\n\t\tDescription: `Move a file or directory from »src« to »dst.«\n\n If »dst« already exists and is a file, it gets overwritten with »src«.\n If »dst« already exists and is a directory, »basename(src)« is created inside,\n (if the file inside does not exist yet)\n\n It's not allowed to move a directory into itself.\n This includes moving the root directory.\n`,\n\t},\n\t\"cp\": {\n\t\tUsage: \"Copy a file or directory from »src« to »dst«\",\n\t\tArgsUsage: \" \",\n\t\tComplete: completeBrigPath(true, true),\n\t\tDescription: `Copy a file or directory from »src« to »dst«.\n\n The semantics are the same as for »brig mv«, except that »cp« does not remove »src«.\n`,\n\t},\n\t\"edit\": {\n\t\tUsage: \"Edit a file in place with $EDITOR\",\n\t\tArgsUsage: \"\",\n\t\tComplete: completeBrigPath(true, false),\n\t\tDescription: `Convenience command to read the file at »path« and display it in $EDITOR.\n\n Once $EDITOR quits, the file is saved back.\n\n If $EDITOR is not set, nano is assumed (I cried a little).\n If nano is not installed this command will fail and you neet to set $EDITOR>\n\n`,\n\t},\n\t\"daemon\": {\n\t\tUsage: \"Daemon management commands.\",\n\t\tComplete: completeSubcommands,\n\t\tDescription: `Commands to manually start or stop the daemon.\n\n The daemon process is normally started whenever you issue the first command\n (like »brig init« or later on a »brig ls«). This command will start it for\n you in the background. Therefore it is seldom useful to use any of those\n commands - unless you are debugging brig.\n`,\n\t},\n\t\"daemon.launch\": {\n\t\tUsage: \"Start the daemon process in the foreground\",\n\t\tComplete: completeArgsUsage,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"trace,t\",\n\t\t\t\tUsage: \"Create tracing output suitable for `go tool trace`\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"s,log-to-stdout\",\n\t\t\t\tUsage: \"Log all messages to stdout instead of syslog\",\n\t\t\t},\n\t\t},\n\t\tDescription: `Start the dameon process in the foreground.\n\n\nEXAMPLES:\n\n $ brig daemon quit # Shut down any previous daemon.\n $ brig daemon launch -s # Start in foreground and log to stdout.\n`,\n\t},\n\t\"daemon.quit\": {\n\t\tUsage: \"Quit a running daemon process\",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: `Quit a running daemon process.\n\n If no daemon process is running, it will tell you.\n`,\n\t},\n\t\"daemon.ping\": {\n\t\tUsage: \"Check if the daemon is running and reachable\",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: `Send up to 100 ping packages to the daemon\n and also print the roundtrip time for each.\n`,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.IntFlag{\n\t\t\t\tName: \"c,count\",\n\t\t\t\tUsage: \"How many times to ping the daemon\",\n\t\t\t\tValue: 10,\n\t\t\t},\n\t\t},\n\t},\n\t\"config\": {\n\t\tUsage: \"View and modify config options.\",\n\t\tComplete: completeSubcommands,\n\t\tDescription: `Commands for getting, setting and listing configuration values.\n\n Each config key is a dotted path (»a.b.c«), associated with one key. These\n configuration values can help you to fine tune the behaviour of brig. In contrast\n to many other programs the config is applied immediately after setting it (where possible).\n Furthermore, each config key will describe itself and tell you if it needs a restart.\n\n For more details on each config value, type 'brig config ls'.\n\n Without further arguments »brig cfg« is a shortcut for »brig cfg ls«.\n`,\n\t},\n\t\"config.get\": {\n\t\tUsage: \"Get a specific config key\",\n\t\tComplete: completeArgsUsage,\n\t\tArgsUsage: \"\",\n\t\tDescription: `Show the current value of a key`,\n\t},\n\t\"config.doc\": {\n\t\tUsage: \"Show the docs for this config key\",\n\t\tComplete: completeArgsUsage,\n\t\tArgsUsage: \"\",\n\t\tDescription: `For each config key a few metadata entries are assigned.\n\nThis includes a string describing the usage, the default value and an indicator\nif the service needs a restart when setting the value.\n\n`,\n\t},\n\t\"config.set\": {\n\t\tUsage: \"Set a specific config key to a new value.\",\n\t\tComplete: completeArgsUsage,\n\t\tArgsUsage: \" \",\n\t\tDescription: `Set the value at »key« to »value«.\n\n Some config values have associated validators that will tell you if a value is not allowed.\n Also you will be warned if the config key requires a restart.\n`,\n\t},\n\t\"config.list\": {\n\t\tUsage: \"List all existing config keys\",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: `List all existing config keys.`,\n\t},\n\t\"fstab\": {\n\t\tUsage: \"Manage mounts that will be mounted on startup of the daemon.\",\n\t\tDescription: \"This is the conceptual equivalent of the normal fstab(5).\",\n\t},\n\t\"fstab.add\": {\n\t\tUsage: \"Add a new mount entry to fstab\",\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"r,readonly\",\n\t\t\t\tUsage: \"Create the filesystem as readonly.\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"offline,o\",\n\t\t\t\tUsage: \"Error out on files that are only remotely available.\",\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"x,root\",\n\t\t\t\tUsage: \"Specify a root directory other than »/«.\",\n\t\t\t},\n\t\t},\n\t},\n\t\"fstab.remove\": {\n\t\tUsage: \"Remove a mount from fstab.\",\n\t},\n\t\"fstab.list\": {\n\t\tUsage: \"List all items in the filesystem table.\",\n\t\tFlags: []cli.Flag{\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"format,f\",\n\t\t\t\tUsage: \"Format the output according to a template.\",\n\t\t\t},\n\t\t},\n\t},\n\t\"fstab.apply\": {\n\t\tUsage: \"Sync the reality with the mounts in fstab.\",\n\t\tDescription: \"Mounts and unmounts directories as necessary.\",\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"u,unmount\",\n\t\t\t\tUsage: \"Unmount all mounts in the filesystem table.\",\n\t\t\t},\n\t\t},\n\t},\n\t\"mount\": {\n\t\tUsage: \"Mount the contents of brig as FUSE filesystem to »mount_path«.\",\n\t\tArgsUsage: \"\",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: `Show the all files and directories inside a normal\n directory. This directory is powered by a userspace filesystem which\n allows you to read and edit data like you are use to from from normal\n files. It is compatible to existing tools and allows brig to interact\n with filebrowsers, video players and other desktop tools.\n\n It is possible to have more than one mount. They will show the same content.\n\nCAVEATS\n\n Editing large files will currently eat big amounts of memory, proportional\n to the size of the file. We advise you to use normal commands like »brig\n cat« and »brig stage« until this is fixed.\n\n At this time, the filesystem also not very robust to files that timeout or\n error out otherwise. Consider this feature to be experimental while this has\n not been worked upon.\n `,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"no-mkdir\",\n\t\t\t\tUsage: \"Do not create the mount directory if it does not exist\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"r,readonly\",\n\t\t\t\tUsage: \"Create the filesystem as readonly\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"offline,o\",\n\t\t\t\tUsage: \"Error out on files that are only remotely available.\",\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"x,root\",\n\t\t\t\tUsage: \"Create the filesystem as readonly\",\n\t\t\t},\n\t\t},\n\t},\n\t\"unmount\": {\n\t\tUsage: \"Unmount a previously mounted directory\",\n\t\tArgsUsage: \"\",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: `Unmount a previously mounted directory.\n\n All mounts get automatically unmounted once the daemon shuts down.\n In case the daemon crashed or failed to unmount, you can manually\n use this command to reclaim the mount point:\n\n $ fusermount -u -z /path/to/mount\n`,\n\t},\n\t\"version\": {\n\t\tUsage: \"Show the version of brig and IPFS\",\n\t\tComplete: completeArgsUsage,\n\t\tDescription: `Show the version of brig and IPFS.\n\n This includes the client and server version of brig.\n These two values should be ideally exactly the same to avoid problems.\n\n Apart from that, the version of IPFS is shown here.\n\n If available, also the git rev is included. This is useful to get the exact\n state of the software in case of problems.\n\n Additionally the build time of the binary is shown.\n Please include this information when reporting a bug.\n`,\n\t},\n\t\"gc\": {\n\t\tUsage: \"Trigger the garbage collector\",\n\t\tComplete: completeArgsUsage,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"aggressive,a\",\n\t\t\t\tUsage: \"Also run the garbage collector on all file systems immediately\",\n\t\t\t},\n\t\t},\n\t\tDescription: `Manually trigger the garbage collector.\n\n Strictly speaking there are two garbage collectors in the system. The\n garbage collector of IPFS cleans up all unpinned files from local storage.\n This still means that the objects referenced there can be retrieved from\n other network nodes, but not locally anymore. This might save alot of space.\n\n The other garbage collector is not very important to the user and cleans up\n unused references inside of the metadata store. It is only run if you pass\n »--aggressive«.\n`,\n\t},\n\t\"docs\": {\n\t\tUsage: \"Open the online documentation in your default web browser.\",\n\t},\n\t\"trash\": {\n\t\tUsage: \"Control the trash bin contents.\",\n\t\tDescription: `\n\n The trash bin is a convenience interface to list and restore deleted files.\n It will list all files that were deleted and were not overwritten by other files.\n `,\n\t},\n\t\"trash.list\": {\n\t\tUsage: \"List all items in the trash bin.\",\n\t},\n\t\"trash.undelete\": {\n\t\tUsage: \"Restore a path from the trashbin.\",\n\t},\n\t\"gateway\": {\n\t\tUsage: \"Control the HTTP/S gateway service.\",\n\t\tDescription: `The gateway serves a UI and download endpoints over a browser.\n This enables users that do not use brig directly to still browse, edit and download files\n For having access to the gateway, users need to be created. By default no users are created.\n Create an »admin« user (password is also »admin«) with this command:\n\n $ brig gw user add admin admin --role-admin\n\n Most of the gateway is configured exclusively via config variables. Please\n refer to the individual config keys for more information (they all start\n with »gateway.«). The »brig gw status« command will also give you a nice, readable\n overview of what the current state is and how you can improve it.\n`,\n\t},\n\t\"gateway.start\": {\n\t\tUsage: \"Start the gateway.\",\n\t\tDescription: `\n It is recommended to check the state with a »brig gw status« afterwards.\n This will give you important hints if something went wrong or needs attention.\n`,\n\t},\n\t\"gateway.stop\": {\n\t\tUsage: \"Stop the gateway.\",\n\t},\n\t\"gateway.status\": {\n\t\tUsage: \"Print a diagnostic report on the status of the gateway.\",\n\t},\n\t\"gateway.url\": {\n\t\tUsage: \"Helper to print the URL to a named file or directory.\",\n\t},\n\t\"gateway.user\": {\n\t\tUsage: \"Control the user account that can access the HTTP gateway.\",\n\t},\n\t\"gateway.user.add\": {\n\t\tUsage: \"Add a new gateway user.\",\n\t\tArgsUsage: \" [ ]\",\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"role-admin,a\",\n\t\t\t\tUsage: \"Add this user as admin (short for »-r 'fs.view,fs.edit,fs.download,remotes.view,remotes.edit'«)\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"role-editor,b\",\n\t\t\t\tUsage: \"Add this user as collaborator (short for »-r 'fs.view,fs.edit,fs.download,remotes.view'«)\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"role-collaborator,c\",\n\t\t\t\tUsage: \"Add this user as collaborator (short for »-r 'fs.view,fs.edit,fs.download'«)\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"role-viewer,d\",\n\t\t\t\tUsage: \"Add this user as viewer (short for »-r 'fs.view,fs.download'«)\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"role-link-only,e\",\n\t\t\t\tUsage: \"Add this user as linker (short for »-r 'fs.download'«)\",\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"rights,r\",\n\t\t\t\tUsage: \"Comma separated list of rights of this user.\",\n\t\t\t},\n\t\t},\n\t\tDescription: `\n The rights are as follows:\n\n fs.view: View and list all files.\n fs.edit: Edit and create new files.\n fs.download: Download file content.\n remotes.view: View the remotes tab.\n remotes.edit: Edit the remotes tab.\n\n If the folder list is empty, this user can access all files.\n If it is non-empty, the user can only access the files including and below all folders.\n`,\n\t},\n\t\"gateway.user.remove\": {\n\t\tUsage: \"Remove a gateway user by its name.\",\n\t},\n\t\"gateway.user.list\": {\n\t\tUsage: \"List all gateway users.\",\n\t\tFlags: []cli.Flag{\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"format,f\",\n\t\t\t\tUsage: \"Format the output by a template.\",\n\t\t\t},\n\t\t},\n\t\tDescription: `\n List all gateway users.\n\n The keys accepted by »--format« are:\n\n - Name: Name of the user.\n - PasswordHash: Hashed password.\n - Salt: Salt of the password.\n - Folders: A list of folders this users may access (might be empty).\n - Rights: A list of rights this users has (might be empty).\n`,\n\t},\n\t\"pack-repo\": {\n\t\tArgsUsage: \"\",\n\t\tDescription: `\n Pack a repo into an encrypted tar archive.\n\n This is mainly useful to lock the repository after using it.\n The encryption key is derived from the password that you either...\n\n * ...enter on stdin.\n * ...specify with --password-command.\n * ...specify with --password-file.\n\n If you move a brig repository between computers or if you use brig\n in an untrusted environment, then this command is for you.\n\n By default, the archive is written next to the repository as\n »$BRIG_PATH.repopack«. If --no-remove is specified the repository\n is not removed upon successful completion.\n\nEXAMPLES:\n\n # Pack a repository, read password from 'pass' and write to usb stick.\n # Also removes the original repository!\n brig --repo /tmp/repo pack-repo \\\n /mnt/usb/brig.repopack \\\n --password-command \"pass my/password/path\"\n`,\n\t\tUsage: \"Create an encrypted archive of the brig repo.\",\n\t\tFlags: []cli.Flag{\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"password-command,p\",\n\t\t\t\tUsage: \"Execute this command to get the password from its stdout.\",\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"password-file,P\",\n\t\t\t\tUsage: \"Read this file to get the password.\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"no-remove,n\",\n\t\t\t\tUsage: \"Do not remove the repository after successfully packing\",\n\t\t\t},\n\t\t},\n\t},\n\t\"unpack-repo\": {\n\t\tArgsUsage: \"\",\n\t\tDescription: `\n The unpack-repo is the inverse of the pack-repo command.\n\nEXAMPLES:\n\n # Unpack a repository from an usb stick and write to a location\n # of your choice. Password is read from 'pass'. The archive is\n # removed upon successful completion.\n\t#\n\t# Specifying --repo is not necessary, but can be used to specify\n\t# where the repository should be unpacked to.\n brig --repo /tmp/repo unpack-repo \\\n /mnt/usb/brig.repopack \\\n --password-command \"pass my/password/path\"\n`,\n\t\tUsage: \"Unpack an encrypted archive of a brig repo.\",\n\t\tFlags: []cli.Flag{\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"password-command,p\",\n\t\t\t\tUsage: \"Execute this command to get the password from its stdout.\",\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"password-file,P\",\n\t\t\t\tUsage: \"Read this file to get the password.\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"no-remove,n\",\n\t\t\t\tUsage: \"Do not remove the archive after successfully unpacking\",\n\t\t\t},\n\t\t},\n\t},\n\t\"debug\": {\n\t\tUsage: \"Various debbugging utilities. Use with care.\",\n\t},\n\t\"debug.decode-stream\": {\n\t\tUsage: \"Decode a brig stream\",\n\t\tFlags: []cli.Flag{\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"key\",\n\t\t\t\tUsage: \"What key to use for encryption (base58 encoded)\",\n\t\t\t\tValue: \"4F7BsTMVPKFshM1MwLf6y23cid6fL3xMpazVoF9krzUw\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"raw\",\n\t\t\t\tUsage: \"Specify if this is a raw stream\",\n\t\t\t},\n\t\t},\n\t},\n\t\"debug.encode-stream\": {\n\t\tUsage: \"Encode a brig stream\",\n\t\tFlags: []cli.Flag{\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"key\",\n\t\t\t\tUsage: \"What key to use for encryption (base58 encoded)\",\n\t\t\t\tValue: \"4F7BsTMVPKFshM1MwLf6y23cid6fL3xMpazVoF9krzUw\",\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"encryption\",\n\t\t\t\tUsage: \"What encryption type to use\",\n\t\t\t\tValue: string(hints.EncryptionAES256GCM),\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"compression,c\",\n\t\t\t\tUsage: \"What compression algorithm to use\",\n\t\t\t\tValue: string(hints.CompressionGuess),\n\t\t\t},\n\t\t},\n\t},\n\t\"debug.ten-source\": {\n\t\tUsage: \"Produce a repeating stream of 0 to 9 numbers of a given size\",\n\t\tFlags: []cli.Flag{\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"size,s\",\n\t\t\t\tUsage: \"Size of the stream in bytes (can be followed by a multipler K, M, or G)\",\n\t\t\t\tValue: \"256M\",\n\t\t\t},\n\t\t},\n\t},\n\t\"debug.ten-sink\": {\n\t\tUsage: \"Check if the stream received over stdin is what ten-source produced\",\n\t\tFlags: []cli.Flag{\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"size,s\",\n\t\t\t\tUsage: \"Expected size of the stream in bytes (can be followed by a multipler K, M, or G)\",\n\t\t\t\tValue: \"256M\",\n\t\t\t},\n\t\t},\n\t},\n\t\"debug.fusemock\": {\n\t\tUsage: \"Start a fuse mock server (use only if you know what you do)\",\n\t\tDescription: \"Hint: You don't.\",\n\t\tFlags: []cli.Flag{\n\t\t\t// NOTE: Changing things need to be also changed in fuse/fusetest.\n\t\t\t// We rely on the argument names there.\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"mount-path,m\",\n\t\t\t\tUsage: \"Path to fuse mount\",\n\t\t\t\tValue: \"\",\n\t\t\t\tRequired: true,\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"catfs-path,c\",\n\t\t\t\tUsage: \"Where to store metadata\",\n\t\t\t\tValue: \"\",\n\t\t\t\tRequired: true,\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"url\",\n\t\t\t\tUsage: \"What compression hint to specify ('*' for all)\",\n\t\t\t\tValue: \"unix:/tmp/fuse-mock.socket\",\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"ipfs-path-or-multiaddr,i\",\n\t\t\t\tUsage: \"Path to IPFS, if you want to use it. Empty for memory only.\",\n\t\t\t\tValue: \"\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"mount-offline,o\",\n\t\t\t\tUsage: \"Mount offline\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"mount-ro,r\",\n\t\t\t\tUsage: \"Mount read-only\",\n\t\t\t},\n\t\t},\n\t},\n\t\"debug.iobench\": {\n\t\tUsage: \"Benchmark I/O on your system\",\n\t\tDescription: `\nThis is a benchmark utility for the streaming system at the heart of brig. If\ninvoked with no argument, it will attempt to do all possible benchmarks. All\nbenchmarks are run in /tmp under the assumption that a tmpfs is located there.\nIf so, you might want to double check that you have enough memory. How much\nis enough depends on many factors, just try it out. :)\n\nIndividual benchmarks can be selected using the '--bench' parameter. It can be\nspecified several times. Check 'brig debug iobench ls' for a full list of\npossible benchmarks.\n\nEach benchmark is run '--sample' times and uses an artificial input with\n'--size' bytes. The input type can be selected by appending one of the\nfollowing names to the benchmark names separated by a colon (':'):\n\n\t* ten: easy to compress data.\n\t* mixed: easy to compress data that is mixed with streaks of hard to\n\t compress data. The ratio of easy to hard is roughly 50%.\n\t* random: Pseudo random data. Terrible to compress.\n\nBy default each benchmark is run several times for each possible hint combination.\nIf you do not want this, specify the desired algorithm using the '--compression'\nand '--encryption' parameters.\n\nIf you want to parse the output of the benchmark, you can use '--json'.\n`,\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"json,j\",\n\t\t\t\tUsage: \"Output the benchmark results as parsable JSON\",\n\t\t\t},\n\t\t\tcli.StringSliceFlag{\n\t\t\t\tName: \"bench,b\",\n\t\t\t\tUsage: \"Which benchmarks to be run (can be given multiple times)\",\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"size,s\",\n\t\t\t\tUsage: \"How big the testdata should be\",\n\t\t\t\tValue: \"256M\",\n\t\t\t},\n\t\t\tcli.IntFlag{\n\t\t\t\tName: \"samples,S\",\n\t\t\t\tUsage: \"How many time to run each benchmark\",\n\t\t\t\tValue: 3,\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"compression,c\",\n\t\t\t\tUsage: \"What compression hint to specify ('*' for all)\",\n\t\t\t\tValue: \"*\",\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"encryption,e\",\n\t\t\t\tUsage: \"What encryption hint to specify ('*' for all)\",\n\t\t\t\tValue: \"*\",\n\t\t\t},\n\t\t},\n\t},\n\t\"debug.iobench.list\": {\n\t\tUsage: \"Just list all benchmark names\",\n\t},\n\t\"debug.pprof-port\": {\n\t\tUsage: \"Print the pprof port of the daemon.\",\n\t\tDescription: `\n This is useful if there is a performance issue (high cpu consumption in idle e.g.).\n See here for some examples of what you can do: https://golang.org/pkg/net/http/pprof\n\nEXAMPLES:\n\n # Show a graph with a cpu profile of the last 30s:\n go tool pprof -web \"http://localhost:$(brig d p)/debug/pprof/profile?seconds=30\"\n`,\n\t},\n\t\"hints\": {\n\t\tUsage: \"Manage hints for file or directories\",\n\t\tDescription: fmt.Sprintf(`\n\n Hints can be used to change the default behavior for brig.\n You can for example use it to change the default encryption algorithm\n for certain files. Hints are always associated to a path. If a hint is\n set to a path where a directory is located, then all files in it inherit\n this hint - except there is another hint somewhere lower in the hierarchy.\n\n Note that hints are only applied on the next file change. Files that have\n differing settings will not be affected by changing a hint. If you want\n an immediate effect you should use »brig hints set --recode «, or,\n if you want to do it a later point, »brig hints recode «.\n\n The available compression algorithms are:\n\n%s\n\n The available encryption algorithms are:\n\n%s\n\nEXAMPLES:\n\n $ brig mkdir /public\n $ echo \"meow\" | brig stage --stdin /public/cat-meme.png\n $ brig hints set /public --compression none --encryption none\n $ brig hints\n PATH ENCRYPTION COMPRESSION\n / aes256gcm guess\n /public none none\n # If a file could be streamed by »ipfs cat« alone,\n # then the »IsRaw« attribute is true.\n $ brig info --format '{{ .IsRaw }}' /public/cat-meme.png\n`, compressionHintsToBullets(), encryptionHintsToBullets()),\n\t},\n\t\"hints.set\": {\n\t\tUsage: \"Set a hint for a file or directory\",\n\t\tDescription: \"See help of »brig hints«\",\n\t\tArgsUsage: \"\",\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"recode,r\",\n\t\t\t\tUsage: \"Recode the stream immediately.\",\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"compression,c\",\n\t\t\t\tUsage: \"What compression algorithm to use for this hint\",\n\t\t\t},\n\t\t\tcli.StringFlag{\n\t\t\t\tName: \"encryption,e\",\n\t\t\t\tUsage: \"What encryption algorithm to use for this hint\",\n\t\t\t},\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"force,f\",\n\t\t\t\tUsage: \"Also create hint if there is no such file or directory\",\n\t\t\t},\n\t\t},\n\t},\n\t\"hints.list\": {\n\t\tUsage: \"List all existing hints.\",\n\t\tDescription: \"See help of »brig hints«\",\n\t},\n\t\"hints.remove\": {\n\t\tArgsUsage: \"\",\n\t\tUsage: \"Remove an existing hint.\",\n\t},\n\t\"hints.recode\": {\n\t\tArgsUsage: \"[]\",\n\t\tUsage: \"Recode the streams in . If no path given all files are recoded.\",\n\t},\n\t\"bug\": {\n\t\tUsage: \"Print a template for bug reports.\",\n\t\tFlags: []cli.Flag{\n\t\t\tcli.BoolFlag{\n\t\t\t\tName: \"stdout,s\",\n\t\t\t\tUsage: \"Always print the report to stdout; do not open a browser\",\n\t\t\t},\n\t\t},\n\t},\n}\n\nfunc injectHelp(cmd *cli.Command, path string) {\n\thelp, ok := helpTexts[path]\n\tif !ok {\n\t\tdie(fmt.Sprintf(\"bug: no such help entry: %v\", path))\n\t}\n\n\tcmd.Usage = help.Usage\n\tcmd.ArgsUsage = help.ArgsUsage\n\tcmd.Description = help.Description\n\tcmd.BashComplete = help.Complete\n\tcmd.Flags = help.Flags\n}\n\nfunc translateHelp(cmds []cli.Command, prefix []string) {\n\tfor idx := range cmds {\n\t\tpath := append(append([]string{}, prefix...), cmds[idx].Name)\n\t\tinjectHelp(&cmds[idx], strings.Join(path, \".\"))\n\t\ttranslateHelp(cmds[idx].Subcommands, path)\n\t}\n}\n\n// TranslateHelp fills in the usage and description for each command.\n// This is separated from the command definition to make things more readable,\n// and separate logic from the (lengthy) documentation.\nfunc TranslateHelp(cmds []cli.Command) []cli.Command {\n\ttranslateHelp(cmds, nil)\n\treturn cmds\n}\n\n// handleOpenHelp opens the online documentation a webbrowser.\nfunc handleOpenHelp(ctx *cli.Context) error {\n\turl := \"https://brig.readthedocs.org\"\n\tif err := webbrowser.Open(url); err != nil {\n\t\tfmt.Printf(\"could not open browser for you: %v\\n\", err)\n\t\tfmt.Printf(\"Please open this link yourself:\\n\\n\\t%s\\n\", url)\n\t} else {\n\t\tfmt.Printf(\"A new tab was opened in your browser.\\n\")\n\t}\n\n\treturn nil\n}\n" }, { "path": "cmd/init.go", "content": "package cmd\n\nimport (\n\te \"github.com/pkg/errors\"\n\t\"github.com/sahib/brig/repo\"\n\t\"github.com/urfave/cli\"\n)\n\n// Init creates a new brig repository at `basePath` with specified options.\nfunc Init(ctx *cli.Context, ipfsPathOrMultiaddr string, opts repo.InitOptions) error {\n\tif err := repo.Init(opts); err != nil {\n\t\treturn e.Wrapf(err, \"repo-init\")\n\t}\n\n\t// Remember the ipsf connection details,\n\t// so we can start it later.\n\treturn repo.OverwriteConfigKey(\n\t\topts.BaseFolder,\n\t\t\"daemon.ipfs_path_or_url\",\n\t\tipfsPathOrMultiaddr,\n\t)\n}\n" }, { "path": "cmd/inode_other.go", "content": "// +build windows\n\npackage cmd\n\n// indodeString convert file path a hardware dependent string\n// unfortunately on non unix platforms DeviceID and Inode are unavailable\n// so we return back the file path\nfunc inodeString(path string) (string, error) {\n\treturn path, nil\n}\n" }, { "path": "cmd/inode_unix.go", "content": "// +build !windows\n\npackage cmd\n\nimport (\n\t\"fmt\"\n\t\"golang.org/x/sys/unix\"\n)\n\n// indodeString convert file path a hardware dependent string in the form DeviceID/Inode\nfunc inodeString(path string) (string, error) {\n\tvar stat unix.Stat_t\n\tif err := unix.Lstat(path, &stat); err != nil {\n\t\treturn path, err\n\t}\n\ts := fmt.Sprintf(\"%d/%d\", stat.Dev, stat.Ino)\n\treturn s, nil\n}\n" }, { "path": "cmd/iobench.go", "content": "package cmd\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"os\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/dustin/go-humanize\"\n\t\"github.com/sahib/brig/bench\"\n\tlog \"github.com/sirupsen/logrus\"\n\t\"github.com/urfave/cli\"\n)\n\nfunc allBenchmarks() []string {\n\tnames := []string{}\n\n\tfor _, benchName := range bench.BenchmarkNames() {\n\t\tfor _, inputName := range bench.InputNames() {\n\t\t\tnames = append(names, fmt.Sprintf(\"%s:%s\", benchName, inputName))\n\t\t}\n\t}\n\n\treturn names\n}\n\nfunc printStats(s bench.Stats) {\n\tfmt.Println()\n\tfmt.Println(\"Time: \", s.Time.Format(time.RFC3339))\n\tfmt.Println(\"CPU Name: \", s.CPUBrandName)\n\tfmt.Println(\"Logical Cores:\", s.LogicalCores)\n\tfmt.Println(\"Has AESNI: \", yesify(s.HasAESNI))\n\tfmt.Println()\n}\n\ntype benchmarkRun struct {\n\tStats bench.Stats `json:\"stats\"`\n\tResults []bench.Result `json:\"results\"`\n}\n\nfunc handleIOBench(ctx *cli.Context) error {\n\trun := benchmarkRun{\n\t\tStats: bench.FetchStats(),\n\t}\n\n\tbenchmarks := ctx.StringSlice(\"bench\")\n\tif len(benchmarks) == 0 {\n\t\tlog.Infof(\"running all benchmarks...\")\n\t\tbenchmarks = allBenchmarks()\n\t}\n\n\tisJSON := ctx.Bool(\"json\")\n\tif !isJSON {\n\t\tprintStats(run.Stats)\n\t}\n\n\tinputSize, err := humanize.ParseBytes(ctx.String(\"size\"))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tsamples := ctx.Int(\"samples\")\n\tif samples <= 0 {\n\t\treturn fmt.Errorf(\"samples must be at least 1\")\n\t}\n\n\tlog.SetLevel(log.WarnLevel)\n\n\tcfgs := []bench.Config{}\n\tfor _, benchmark := range benchmarks {\n\t\tbenchSplit := strings.SplitN(benchmark, \":\", 2)\n\n\t\tbenchInput := \"ten\"\n\t\tbenchName := benchSplit[0]\n\t\tif len(benchSplit) >= 2 {\n\t\t\tbenchInput = benchSplit[1]\n\t\t}\n\n\t\tcfgs = append(cfgs, bench.Config{\n\t\t\tBenchName: benchName,\n\t\t\tInputName: benchInput,\n\t\t\tSize: inputSize,\n\t\t\tSamples: samples,\n\t\t\tEncryption: ctx.String(\"encryption\"),\n\t\t\tCompression: ctx.String(\"compression\"),\n\t\t})\n\t}\n\n\tvar baselineTiming time.Duration\n\tvar lastSection string\n\n\terr = bench.Benchmark(cfgs, func(result bench.Result) {\n\t\tsection := fmt.Sprintf(\n\t\t\t\"%s:%s\",\n\t\t\tresult.Config.InputName,\n\t\t\tresult.Config.BenchName,\n\t\t)\n\n\t\tif section != lastSection {\n\t\t\tif !isJSON {\n\t\t\t\tdrawHeading(section)\n\t\t\t}\n\n\t\t\t// First in list is always the none-none benchmark.\n\t\t\tbaselineTiming = result.Took\n\t\t\tlastSection = section\n\t\t}\n\n\t\tif !isJSON {\n\t\t\tdrawBench(result, baselineTiming, inputSize)\n\t\t}\n\n\t\trun.Results = append(run.Results, result)\n\t})\n\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif isJSON {\n\t\tenc := json.NewEncoder(os.Stdout)\n\t\tenc.SetIndent(\"\", \" \")\n\t\tenc.Encode(run)\n\t}\n\n\treturn nil\n}\n\nfunc drawHeading(heading string) {\n\tfmt.Println()\n\tfmt.Println(heading)\n\tfmt.Println(strings.Repeat(\"=\", len(heading)))\n\tfmt.Println()\n}\n\nfunc drawBench(result bench.Result, ref time.Duration, inputSize uint64) {\n\tfmt.Printf(\n\t\t\"%-45s %9.2f MB/s %20s %8.2f%% %6d allocs %8.2f%% zipped\\n\",\n\t\tresult.Name,\n\t\tresult.Throughput,\n\t\tfmt.Sprintf(\n\t\t\t\"%.2fMB/%v\",\n\t\t\tfloat64(inputSize)/1000/1000,\n\t\t\tresult.Took.Round(time.Millisecond),\n\t\t),\n\t\t100*float64(ref)/float64(result.Took),\n\t\tresult.Allocs,\n\t\tresult.CompressionRate*100,\n\t)\n}\n\nfunc handleIOBenchList(ctx *cli.Context) error {\n\tfor _, name := range allBenchmarks() {\n\t\tfmt.Println(name)\n\t}\n\n\treturn nil\n}\n" }, { "path": "cmd/log.go", "content": "package cmd\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"strings\"\n\n\t\"github.com/urfave/cli\"\n)\n\nfunc logVerbose(ctx *cli.Context, format string, args ...interface{}) {\n\tif !ctx.GlobalBool(\"verbose\") {\n\t\treturn\n\t}\n\n\tif !strings.HasSuffix(format, \"\\n\") {\n\t\tformat = format + \"\\n\"\n\t}\n\n\tfmt.Fprintf(os.Stderr, \"-- \"+format, args...)\n}\n" }, { "path": "cmd/net_handlers.go", "content": "package cmd\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"os\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/fatih/color\"\n\t\"github.com/sahib/brig/cmd/tabwriter\"\n\n\t\"github.com/sahib/brig/client\"\n\t\"github.com/urfave/cli\"\n\tyml \"gopkg.in/yaml.v2\"\n)\n\nfunc handleOffline(ctx *cli.Context, ctl *client.Client) error {\n\treturn ctl.NetDisconnect()\n}\n\nfunc handleOnline(ctx *cli.Context, ctl *client.Client) error {\n\treturn ctl.NetConnect()\n}\n\nfunc handleIsOnline(ctx *cli.Context, ctl *client.Client) error {\n\tself, err := ctl.Whoami()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif self.IsOnline {\n\t\tfmt.Println(color.GreenString(\"online\"))\n\t} else {\n\t\tfmt.Println(color.RedString(\"offline\"))\n\t}\n\n\treturn nil\n}\n\nfunc handleRemoteList(ctx *cli.Context, ctl *client.Client) error {\n\tif ctx.Bool(\"offline\") {\n\t\treturn handleRemoteListOffline(ctx, ctl)\n\t}\n\n\treturn handleRemoteListOnline(ctx, ctl)\n}\n\nfunc nFoldersToIcon(nFolders int) string {\n\tif nFolders == 0 {\n\t\treturn color.GreenString(\"*\")\n\t}\n\n\treturn color.YellowString(fmt.Sprintf(\"%d\", nFolders))\n}\n\nfunc handleRemoteListOffline(ctx *cli.Context, ctl *client.Client) error {\n\tremotes, err := ctl.RemoteLs()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"remote ls: %v\", err)\n\t}\n\n\tif ctx.IsSet(\"format\") {\n\t\ttmpl, err := readFormatTemplate(ctx)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfor _, remote := range remotes {\n\t\t\tif err := tmpl.Execute(os.Stdout, remote); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t}\n\n\tif len(remotes) == 0 {\n\t\tfmt.Println(\"No remotes yet. Use `brig remote add »user« »fingerprint«` to add some.\")\n\t\treturn nil\n\t}\n\n\ttabW := tabwriter.NewWriter(\n\t\tos.Stdout, 0, 0, 2, ' ',\n\t\ttabwriter.StripEscape,\n\t)\n\n\tfmt.Fprintln(tabW, \"NAME\\tFINGERPRINT\\tAUTO-UPDATE\\tACCEPT PUSH\\tCONFLICT STRATEGY\\tFOLDERS\\t\")\n\n\tfor _, remote := range remotes {\n\t\tcs := remote.ConflictStrategy\n\t\tif cs == \"\" {\n\t\t\tcs = \"marker\"\n\t\t}\n\n\t\tfmt.Fprintf(\n\t\t\ttabW,\n\t\t\t\"%s\\t%s\\t%s\\t%s\\t%s\\t%s\\n\",\n\t\t\tremote.Name,\n\t\t\tremote.Fingerprint,\n\t\t\tyesOrNo(remote.AutoUpdate),\n\t\t\tyesOrNo(remote.AcceptPush),\n\t\t\tcs,\n\t\t\tnFoldersToIcon(len(remote.Folders)),\n\t\t)\n\t}\n\n\treturn tabW.Flush()\n}\n\nfunc handleRemoteListOnline(ctx *cli.Context, ctl *client.Client) error {\n\tpeers, err := ctl.RemoteOnlineList()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\ttabW := tabwriter.NewWriter(\n\t\tos.Stdout, 0, 0, 2, ' ',\n\t\ttabwriter.StripEscape,\n\t)\n\n\tif len(peers) == 0 {\n\t\tfmt.Println(\"Remote list is empty. Nobody there to ping.\")\n\t\treturn nil\n\t}\n\n\tif !ctx.IsSet(\"format\") {\n\t\tfmt.Fprintln(tabW, \"NAME\\tFINGERPRINT\\tROUNDTRIP\\tONLINE\\tAUTHENTICATED\\tLASTSEEN\\tAUTO-UPDATE\\tACCEPT PUSH\\tCONFLICT STRATEGY\\tFOLDERS\\t\")\n\t}\n\n\ttmpl, err := readFormatTemplate(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor _, status := range peers {\n\t\tif tmpl != nil {\n\t\t\trmt := client.Remote{\n\t\t\t\tFingerprint: status.Remote.Fingerprint,\n\t\t\t\tName: status.Remote.Name,\n\t\t\t}\n\n\t\t\tif err := tmpl.Execute(os.Stdout, rmt); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tcontinue\n\t\t}\n\n\t\troundtrip := status.Roundtrip.String()\n\t\tisOnline := color.GreenString(\"✔ \")\n\n\t\tif status.Err != nil {\n\t\t\tisOnline = color.RedString(\"✘ \" + status.Err.Error())\n\t\t\troundtrip = \"∞\"\n\t\t}\n\n\t\tauthenticated := color.RedString(\"✘\")\n\t\tif status.Authenticated {\n\t\t\tauthenticated = color.GreenString(\"✔\")\n\t\t}\n\n\t\tshortFp := \"\"\n\n\t\tsplitFp := strings.SplitN(status.Remote.Fingerprint, \":\", 2)\n\t\tif len(splitFp) > 0 {\n\t\t\tshortAddr := splitFp[0]\n\t\t\tif len(shortAddr) > 12 {\n\t\t\t\tshortAddr = shortAddr[:12]\n\t\t\t}\n\n\t\t\tshortFp += shortAddr\n\t\t}\n\n\t\tif len(splitFp) > 1 {\n\t\t\tshortPubKeyID := splitFp[1]\n\t\t\tif len(shortPubKeyID) > 12 {\n\t\t\t\tshortPubKeyID = shortPubKeyID[:12]\n\t\t\t}\n\n\t\t\tshortFp += \":\"\n\t\t\tshortFp += shortPubKeyID\n\t\t}\n\n\t\tcs := status.Remote.ConflictStrategy\n\t\tif cs == \"\" {\n\t\t\tcs = \"marker\"\n\t\t}\n\n\t\tfmt.Fprintf(\n\t\t\ttabW,\n\t\t\t\"%s\\t%s\\t%s\\t%s\\t%s\\t%s\\t%s\\t%s\\t%s\\t%s\\t\\n\",\n\t\t\tstatus.Remote.Name,\n\t\t\tshortFp,\n\t\t\troundtrip,\n\t\t\tisOnline,\n\t\t\tauthenticated,\n\t\t\tstatus.LastSeen.Format(time.UnixDate),\n\t\t\tyesOrNo(status.Remote.AutoUpdate),\n\t\t\tyesOrNo(status.Remote.AcceptPush),\n\t\t\tcs,\n\t\t\tnFoldersToIcon(len(status.Remote.Folders)),\n\t\t)\n\t}\n\n\treturn tabW.Flush()\n}\n\nconst (\n\tremoteHelpText = `# No remotes yet. Uncomment the next lines for an example:\n# - Name: alice@wonderland.com\n# Fingerprint: QmVA5j2JHPkDTHgZ[...]:SEfXUDeJA1toVnP[...]\n`\n)\n\nfunc remoteListToYml(remotes []client.Remote) ([]byte, error) {\n\tif len(remotes) == 0 {\n\t\t// Provide a helpful description, instead of an empty list.\n\t\treturn []byte(remoteHelpText), nil\n\t}\n\n\treturn yml.Marshal(remotes)\n}\n\nfunc ymlToRemoteList(data []byte) ([]client.Remote, error) {\n\tremotes := []client.Remote{}\n\n\tif err := yml.Unmarshal(data, &remotes); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn remotes, nil\n}\n\nfunc handleRemoteAdd(ctx *cli.Context, ctl *client.Client) error {\n\tremote := client.Remote{\n\t\tName: ctx.Args().Get(0),\n\t\tFingerprint: ctx.Args().Get(1),\n\t\tAutoUpdate: ctx.Bool(\"auto-update\"),\n\t\tConflictStrategy: ctx.String(\"conflict-strategy\"),\n\t\tAcceptPush: ctx.Bool(\"accept-push\"),\n\t}\n\n\tfor _, folder := range ctx.StringSlice(\"folder\") {\n\t\tisReadOnly := false\n\t\tif strings.HasPrefix(folder, \"-\") {\n\t\t\tisReadOnly = true\n\t\t\tfolder = folder[1:]\n\t\t}\n\n\t\tremote.Folders = append(remote.Folders, client.RemoteFolder{\n\t\t\tFolder: folder,\n\t\t\tReadOnly: isReadOnly,\n\t\t})\n\t}\n\n\tif err := ctl.RemoteAddOrUpdate(remote); err != nil {\n\t\treturn fmt.Errorf(\"remote add: %v\", err)\n\t}\n\n\treturn nil\n}\n\nfunc handleRemoteAutoUpdate(ctx *cli.Context, ctl *client.Client) error {\n\tenable := true\n\n\tswitch ctx.Args().First() {\n\tcase \"enable\", \"e\":\n\t\tenable = true\n\tcase \"disable\", \"d\":\n\t\tenable = false\n\tdefault:\n\t\treturn fmt.Errorf(\"please specify 'enable' or 'disable' as first argument\")\n\t}\n\n\tfor _, remoteName := range ctx.Args()[1:] {\n\t\trmt, err := ctl.RemoteByName(remoteName)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\trmt.AutoUpdate = enable\n\t\tif err := ctl.RemoteAddOrUpdate(rmt); err != nil {\n\t\t\treturn fmt.Errorf(\"remote update: %v\", err)\n\t\t}\n\n\t\tif !ctx.Bool(\"no-initial-sync\") {\n\t\t\tif _, err := ctl.Sync(remoteName, true); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc handleRemoteAcceptPush(ctx *cli.Context, ctl *client.Client) error {\n\tenable := true\n\n\tswitch ctx.Args().First() {\n\tcase \"enable\", \"e\":\n\t\tenable = true\n\tcase \"disable\", \"d\":\n\t\tenable = false\n\tdefault:\n\t\treturn fmt.Errorf(\"please specify 'enable' or 'disable' as first argument\")\n\t}\n\n\tfor _, remoteName := range ctx.Args()[1:] {\n\t\trmt, err := ctl.RemoteByName(remoteName)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\trmt.AcceptPush = enable\n\t\tif err := ctl.RemoteAddOrUpdate(rmt); err != nil {\n\t\t\treturn fmt.Errorf(\"remote update: %v\", err)\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc handleRemoteConflictStrategy(ctx *cli.Context, ctl *client.Client) error {\n\tfor _, remoteName := range ctx.Args()[1:] {\n\t\trmt, err := ctl.RemoteByName(remoteName)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\trmt.ConflictStrategy = ctx.Args().First()\n\t\tif err := ctl.RemoteAddOrUpdate(rmt); err != nil {\n\t\t\treturn fmt.Errorf(\"remote update: %v\", err)\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc handleRemoteRemove(ctx *cli.Context, ctl *client.Client) error {\n\tname := ctx.Args().First()\n\tif err := ctl.RemoteRm(name); err != nil {\n\t\treturn fmt.Errorf(\"remote rm: %v\", err)\n\t}\n\n\treturn nil\n}\n\nfunc handleRemoteClear(ctx *cli.Context, ctl *client.Client) error {\n\treturn ctl.RemoteClear()\n}\n\nfunc handleRemoteEdit(ctx *cli.Context, ctl *client.Client) error {\n\tremotes, err := ctl.RemoteLs()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"remote ls: %v\", err)\n\t}\n\n\tdata, err := remoteListToYml(remotes)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"Failed to convert to yml: %v\", err)\n\t}\n\n\t// Launch an editor on the received data:\n\tnewData, err := edit(data, \"yml\")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"Failed to launch editor: %v\", err)\n\t}\n\n\t// Save a few network roundtrips if nothing was changed:\n\tif bytes.Equal(data, newData) {\n\t\tfmt.Println(\"Nothing changed.\")\n\t\treturn nil\n\t}\n\n\tnewRemotes, err := ymlToRemoteList(newData)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif err := ctl.RemoteSave(newRemotes); err != nil {\n\t\treturn fmt.Errorf(\"Saving back remotes failed: %v\", err)\n\t}\n\n\treturn nil\n}\n\nfunc findRemoteForName(ctl *client.Client, name string) (*client.Remote, error) {\n\tremotes, err := ctl.RemoteLs()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor _, remote := range remotes {\n\t\tif remote.Name == name {\n\t\t\treturn &remote, nil\n\t\t}\n\t}\n\n\treturn nil, fmt.Errorf(\"No such remote with this name: %s\", name)\n}\n\nfunc handleRemoteFolderAdd(ctx *cli.Context, ctl *client.Client) error {\n\treturn handleRemoteFolderAddOrReplace(ctx, ctl, false)\n}\n\nfunc handleRemoteFolderSet(ctx *cli.Context, ctl *client.Client) error {\n\treturn handleRemoteFolderAddOrReplace(ctx, ctl, true)\n}\n\nfunc handleRemoteFolderAddOrReplace(ctx *cli.Context, ctl *client.Client, replace bool) error {\n\tremote, err := findRemoteForName(ctl, ctx.Args().First())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tisReadOnly := ctx.Bool(\"read-only\")\n\tconflictStrategy := ctx.String(\"conflict-strategy\")\n\n\tfor _, folder := range ctx.Args().Tail() {\n\t\tif _, err := ctl.Stat(folder); err != nil {\n\t\t\tfmt.Printf(\"warning: »%s« does not seem to exist. That's fine though, just in case you made a typo.\\n\", folder)\n\t\t}\n\n\t\tfolderFound := false\n\n\t\tfor idx, remoteFolder := range remote.Folders {\n\t\t\tif remoteFolder.Folder == folder {\n\t\t\t\tif replace {\n\t\t\t\t\tif ctx.IsSet(\"read-only\") {\n\t\t\t\t\t\tremote.Folders[idx].ReadOnly = true\n\t\t\t\t\t}\n\n\t\t\t\t\tif ctx.IsSet(\"read-write\") {\n\t\t\t\t\t\tremote.Folders[idx].ReadOnly = false\n\t\t\t\t\t}\n\n\t\t\t\t\tif ctx.IsSet(\"conflict-strategy\") {\n\t\t\t\t\t\tremote.Folders[idx].ConflictStrategy = conflictStrategy\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\tfolderFound = true\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\n\t\tif !replace && folderFound {\n\t\t\treturn fmt.Errorf(\"»%s« exists already\", folder)\n\t\t}\n\n\t\tif replace && !folderFound {\n\t\t\treturn fmt.Errorf(\"»%s« does not exist\", folder)\n\t\t}\n\n\t\tif !replace {\n\t\t\tremote.Folders = append(remote.Folders, client.RemoteFolder{\n\t\t\t\tFolder: folder,\n\t\t\t\tReadOnly: isReadOnly,\n\t\t\t\tConflictStrategy: conflictStrategy,\n\t\t\t})\n\t\t}\n\t}\n\n\treturn ctl.RemoteUpdate(*remote)\n}\n\nfunc handleRemoteFolderRemove(ctx *cli.Context, ctl *client.Client) error {\n\tremote, err := findRemoteForName(ctl, ctx.Args().First())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfolderName := ctx.Args().Get(1)\n\tnewFolders := []client.RemoteFolder{}\n\tfor _, folder := range remote.Folders {\n\t\tif string(folder.Folder) == folderName {\n\t\t\tcontinue\n\t\t}\n\n\t\tnewFolders = append(newFolders, folder)\n\t}\n\n\tremote.Folders = newFolders\n\treturn ctl.RemoteUpdate(*remote)\n}\n\nfunc handleRemoteFolderClear(ctx *cli.Context, ctl *client.Client) error {\n\tremote, err := findRemoteForName(ctl, ctx.Args().First())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tremote.Folders = []client.RemoteFolder{}\n\treturn ctl.RemoteUpdate(*remote)\n}\n\nfunc handleRemoteFolderList(ctx *cli.Context, ctl *client.Client) error {\n\tremote, err := findRemoteForName(ctl, ctx.Args().First())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif len(remote.Folders) == 0 {\n\t\tfmt.Println(\"No folders specified. All folders are accessible.\")\n\t\treturn nil\n\t}\n\n\ttabW := tabwriter.NewWriter(os.Stdout, 0, 0, 2, ' ', tabwriter.StripEscape)\n\tfmt.Fprintln(tabW, \"FOLDER\\tREAD ONLY\\tCONFLICT STRATEGY\\t\")\n\n\tfor _, folder := range remote.Folders {\n\t\tfmt.Fprintf(\n\t\t\ttabW,\n\t\t\t\"%s\\t%s\\t%s\\t\\n\",\n\t\t\tfolder.Folder,\n\t\t\tyesOrNo(folder.ReadOnly),\n\t\t\tfolder.ConflictStrategy,\n\t\t)\n\t}\n\n\treturn tabW.Flush()\n}\n\nfunc handleRemoteFolderListAll(ctx *cli.Context, ctl *client.Client) error {\n\tremotes, err := ctl.RemoteLs()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\ttabW := tabwriter.NewWriter(os.Stdout, 0, 0, 2, ' ', tabwriter.StripEscape)\n\tfmt.Fprintln(tabW, \"REMOTE\\tFOLDER\\tREAD ONLY\\tCONFLICT STRATEGY\\t\")\n\n\tfor _, remote := range remotes {\n\t\tfor _, folder := range remote.Folders {\n\t\t\tfmt.Fprintf(\n\t\t\t\ttabW,\n\t\t\t\t\"%s\\t%s\\t%s\\t%s\\t\\n\",\n\t\t\t\tremote.Name,\n\t\t\t\tfolder.Folder,\n\t\t\t\tyesOrNo(folder.ReadOnly),\n\t\t\t\tfolder.ConflictStrategy,\n\t\t\t)\n\t\t}\n\t}\n\n\treturn tabW.Flush()\n}\n\nfunc handleNetLocate(ctx *cli.Context, ctl *client.Client) error {\n\twho := ctx.Args().First()\n\ttimeoutSec, err := parseDuration(ctx.String(\"timeout\"))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Show a progress ticker, since the query might take quite long:\n\tprogressTicker := time.NewTicker(500 * time.Millisecond)\n\tgo func() {\n\t\tnDots := 0\n\t\tfor range progressTicker.C {\n\t\t\tfmt.Printf(\"Scanning%-5s\\r\", strings.Repeat(\".\", nDots+1))\n\t\t\tnDots = (nDots + 1) % 5\n\t\t}\n\t}()\n\n\tcandidateCh, err := ctl.NetLocate(who, ctx.String(\"mask\"), timeoutSec)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"Failed to locate peers: %v\", err)\n\t}\n\n\tsomethingFound := false\n\n\tfor candidate := range candidateCh {\n\t\tif !somethingFound {\n\t\t\tprogressTicker.Stop()\n\t\t\tsomethingFound = true\n\n\t\t\t// We can't use tabwriter here, sine it needs to update in realtime.\n\t\t\t// So we just fake it (badly) with printf-like formatting.\n\t\t\tfmt.Printf(\"%-30s %-10s %s\\n\", \"NAME\", \"TYPE\", \"FINGERPRINT\")\n\t\t}\n\n\t\tfingerprint := candidate.Fingerprint\n\t\tif fingerprint == \"\" {\n\t\t\tfingerprint = candidate.Addr + color.RedString(\" (offline)\")\n\t\t} else {\n\t\t\tfingerprint = color.GreenString(fingerprint)\n\t\t}\n\n\t\tfmt.Printf(\n\t\t\t\"%-30s %-10s %s\\n\",\n\t\t\tcandidate.Name,\n\t\t\tstrings.Join(candidate.Mask, \"|\"),\n\t\t\tfingerprint,\n\t\t)\n\t}\n\n\tif !somethingFound {\n\t\tfmt.Println(\"No results. Maybe nobodoy is online?\")\n\t}\n\n\treturn nil\n}\n\nfunc handleRemotePing(ctx *cli.Context, ctl *client.Client) error {\n\twho := ctx.Args().First()\n\n\tmsg := fmt.Sprintf(\"ping to %s: \", color.MagentaString(who))\n\troundtrip, err := ctl.RemotePing(who)\n\tif err != nil {\n\t\tmsg += color.RedString(\"✘\")\n\t\tmsg += fmt.Sprintf(\" (%v)\", err)\n\t} else {\n\t\tmsg += color.GreenString(\"✔\")\n\t\tmsg += fmt.Sprintf(\" (%3.5fs)\", roundtrip)\n\t}\n\n\tfmt.Println(msg)\n\treturn nil\n}\n\nfunc handlePin(ctx *cli.Context, ctl *client.Client) error {\n\tpath := ctx.Args().First()\n\treturn ctl.Pin(path)\n}\n\nfunc handleUnpin(ctx *cli.Context, ctl *client.Client) error {\n\tpath := ctx.Args().First()\n\treturn ctl.Unpin(path)\n}\n\nfunc handleRepin(ctx *cli.Context, ctl *client.Client) error {\n\troot := \"/\"\n\tif len(ctx.Args()) > 0 {\n\t\troot = ctx.Args().First()\n\t}\n\n\treturn ctl.Repin(root)\n}\n\nfunc handleWhoami(ctx *cli.Context, ctl *client.Client) error {\n\tself, err := ctl.Whoami()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tsplitFingerprint := strings.SplitN(self.Fingerprint, \":\", 2)\n\n\tprintFingerprint := ctx.Bool(\"fingerprint\")\n\tprintName := ctx.Bool(\"name\")\n\tprintAddr := ctx.Bool(\"addr\")\n\tprintKey := ctx.Bool(\"key\")\n\n\tuserName := color.YellowString(self.CurrentUser)\n\townerName := color.GreenString(self.Owner)\n\n\tif !printFingerprint && !printName && !printAddr && !printKey {\n\t\tif self.CurrentUser != self.Owner {\n\t\t\tfmt.Printf(\n\t\t\t\t\"# Note: viewing %s's data currently\\n\",\n\t\t\t\tcolor.YellowString(userName),\n\t\t\t)\n\t\t}\n\n\t\tfmt.Printf(\"- Name: %s\\n\", color.YellowString(self.Owner))\n\t\tfmt.Printf(\" Fingerprint: %s\\n\", self.Fingerprint)\n\n\t\treturn nil\n\t}\n\n\tif printName {\n\t\tfmt.Printf(\"%s\", ownerName)\n\t}\n\n\tif printFingerprint {\n\t\tif printName {\n\t\t\tfmt.Printf(\" \")\n\t\t}\n\t\tfmt.Printf(\"%s\", self.Fingerprint)\n\t}\n\n\tif printAddr {\n\t\tif printName || printFingerprint {\n\t\t\tfmt.Printf(\" \")\n\t\t}\n\n\t\tif len(splitFingerprint) > 0 {\n\t\t\tfmt.Printf(\"%s\", splitFingerprint[0])\n\t\t}\n\t}\n\n\tif printKey {\n\t\tif printName || printFingerprint || printAddr {\n\t\t\tfmt.Printf(\" \")\n\t\t}\n\n\t\tif len(splitFingerprint) > 1 {\n\t\t\tfmt.Printf(\"%s\", splitFingerprint[1])\n\t\t}\n\t}\n\n\tfmt.Printf(\"\\n\")\n\treturn nil\n}\n\nfunc handlePush(ctx *cli.Context, ctl *client.Client) error {\n\tremoteName := ctx.Args().First()\n\treturn ctl.Push(remoteName, ctx.Bool(\"dry-run\"))\n}\n" }, { "path": "cmd/parser.go", "content": "package cmd\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"runtime\"\n\t\"runtime/debug\"\n\t\"runtime/pprof\"\n\t\"strings\"\n\n\t\"github.com/fatih/color\"\n\tisatty \"github.com/mattn/go-isatty\"\n\t\"github.com/sahib/brig/defaults\"\n\tformatter \"github.com/sahib/brig/util/log\"\n\t\"github.com/sahib/brig/version\"\n\tlog \"github.com/sirupsen/logrus\"\n\t\"github.com/urfave/cli\"\n)\n\nfunc init() {\n\tlog.SetOutput(os.Stderr)\n\tlog.SetLevel(log.DebugLevel)\n\n\tvar useColor bool\n\tswitch envVar := os.Getenv(\"BRIG_COLOR\"); envVar {\n\tcase \"\", \"auto\":\n\t\tuseColor = isatty.IsTerminal(os.Stdout.Fd())\n\tcase \"never\":\n\t\tuseColor = false\n\tcase \"always\":\n\t\tuseColor = true\n\tdefault:\n\t\tlog.Warningf(\"Bad value for $BRIG_COLOR: %s, disabling color\", envVar)\n\t\tuseColor = false\n\t}\n\n\t// Only use fancy logging if we print to a terminal:\n\tlog.SetFormatter(&formatter.FancyLogFormatter{\n\t\tUseColors: useColor,\n\t})\n}\n\nfunc formatGroup(category string) string {\n\treturn \"\\n\" + strings.ToUpper(category) + \" COMMANDS\"\n}\n\nfunc memProfile() {\n\tmemPath := os.Getenv(\"BRIG_MEM_PROFILE\")\n\tif memPath == \"\" {\n\t\treturn\n\t}\n\n\tfd, err := os.Create(memPath)\n\tif err != nil {\n\t\tlog.Fatal(\"could not create memory profile: \", err)\n\t}\n\n\tdefer fd.Close()\n\n\truntime.GC()\n\tif err := pprof.WriteHeapProfile(fd); err != nil {\n\t\tlog.Fatal(\"could not write memory profile: \", err)\n\t}\n}\n\nfunc startCPUProfile() *os.File {\n\tcpuPath := os.Getenv(\"BRIG_CPU_PROFILE\")\n\tif cpuPath == \"\" {\n\t\treturn nil\n\t}\n\n\tfd, err := os.Create(cpuPath)\n\tif err != nil {\n\t\tlog.Fatal(\"could not create memory profile: \", err)\n\t}\n\n\truntime.GC()\n\tif err := pprof.StartCPUProfile(fd); err != nil {\n\t\tlog.Fatal(\"could not write memory profile: \", err)\n\t}\n\n\treturn fd\n}\n\nfunc stopCPUProfile(fd *os.File) {\n\tif os.Getenv(\"BRIG_CPU_PROFILE\") == \"\" {\n\t\treturn\n\t}\n\n\tdefer fd.Close()\n\tpprof.StopCPUProfile()\n}\n\n////////////////////////////\n// Commandline definition //\n////////////////////////////\n\n// RunCmdline starts a brig commandline tool.\nfunc RunCmdline(args []string) int {\n\tprofFd := startCPUProfile()\n\tdefer stopCPUProfile(profFd)\n\tdefer memProfile()\n\n\tdebug.SetTraceback(\"all\")\n\n\tapp := cli.NewApp()\n\tapp.Name = \"brig\"\n\tapp.Usage = \"Secure and decentralized file synchronization\"\n\tapp.EnableBashCompletion = true\n\tapp.Version = fmt.Sprintf(\n\t\t\"%s [buildtime: %s] (client version)\",\n\t\tversion.String(),\n\t\tversion.BuildTime,\n\t)\n\tapp.CommandNotFound = commandNotFound\n\tapp.Description = \"brig can be used to securely store, version and synchronize files between many peers.\"\n\n\t// Set global options here:\n\tapp.Before = func(ctx *cli.Context) error {\n\t\tif ctx.Bool(\"no-color\") {\n\t\t\tcolor.NoColor = true\n\t\t}\n\n\t\treturn nil\n\t}\n\n\t// Groups:\n\trepoGroup := formatGroup(\"repository\")\n\twdirGroup := formatGroup(\"working tree\")\n\tvcscGroup := formatGroup(\"version control\")\n\tnetwGroup := formatGroup(\"network\")\n\n\t// Autocomplete all commands, but not their aliases.\n\tapp.BashComplete = func(ctx *cli.Context) {\n\t\tfor _, cmd := range app.Commands {\n\t\t\tfmt.Println(cmd.Name)\n\t\t}\n\t}\n\n\tapp.Flags = []cli.Flag{\n\t\tcli.StringFlag{\n\t\t\tName: \"url,u\",\n\t\t\tUsage: \"URL on where to reach the brig daemon.\",\n\t\t\tEnvVar: \"BRIG_URL\",\n\t\t\tValue: defaults.DaemonDefaultURL(),\n\t\t},\n\t\tcli.StringFlag{\n\t\t\tName: \"repo\",\n\t\t\tUsage: \"Path to the repository. Only has effect for new daemons.\",\n\t\t\tValue: \".\",\n\t\t\tEnvVar: \"BRIG_PATH\",\n\t\t},\n\t\tcli.BoolFlag{\n\t\t\tName: \"verbose,V\",\n\t\t\tUsage: \"Show certain messages during client startup (helpful for debugging)\",\n\t\t},\n\t\tcli.BoolFlag{\n\t\t\tName: \"nodaemon,n\",\n\t\t\tUsage: \"Don't start the daemon automatically.\",\n\t\t},\n\t\tcli.BoolFlag{\n\t\t\tName: \"no-color\",\n\t\t\tUsage: \"Forbid the usage of colors.\",\n\t\t},\n\t}\n\n\tapp.Commands = TranslateHelp([]cli.Command{\n\t\t{\n\t\t\tName: \"init\",\n\t\t\tCategory: repoGroup,\n\t\t\tAction: handleInit,\n\t\t}, {\n\t\t\tName: \"whoami\",\n\t\t\tAliases: []string{\"id\"},\n\t\t\tCategory: netwGroup,\n\t\t\tAction: withDaemon(handleWhoami, true),\n\t\t}, {\n\t\t\tName: \"remote\",\n\t\t\tAliases: []string{\"rmt\", \"r\"},\n\t\t\tCategory: netwGroup,\n\t\t\tSubcommands: []cli.Command{\n\t\t\t\t{\n\t\t\t\t\tName: \"add\",\n\t\t\t\t\tAliases: []string{\"a\", \"set\"},\n\t\t\t\t\tAction: withArgCheck(needAtLeast(2), withDaemon(handleRemoteAdd, true)),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"remove\",\n\t\t\t\t\tAliases: []string{\"rm\"},\n\t\t\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleRemoteRemove, true)),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"list\",\n\t\t\t\t\tAliases: []string{\"ls\"},\n\t\t\t\t\tAction: withDaemon(handleRemoteList, true),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"clear\",\n\t\t\t\t\tAction: withDaemon(handleRemoteClear, true),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"edit\",\n\t\t\t\t\tAction: withDaemon(handleRemoteEdit, true),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"ping\",\n\t\t\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleRemotePing, true)),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"auto-update\",\n\t\t\t\t\tAliases: []string{\"au\"},\n\t\t\t\t\tAction: withArgCheck(needAtLeast(2), withDaemon(handleRemoteAutoUpdate, true)),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"accept-push\",\n\t\t\t\t\tAliases: []string{\"ap\"},\n\t\t\t\t\tAction: withArgCheck(needAtLeast(2), withDaemon(handleRemoteAcceptPush, true)),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"conflict-strategy\",\n\t\t\t\t\tAliases: []string{\"cs\"},\n\t\t\t\t\tAction: withArgCheck(needAtLeast(2), withDaemon(handleRemoteConflictStrategy, true)),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"folder\",\n\t\t\t\t\tAliases: []string{\"fld\", \"f\"},\n\t\t\t\t\tAction: withDaemon(handleRemoteFolderListAll, true),\n\t\t\t\t\tSubcommands: []cli.Command{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"add\",\n\t\t\t\t\t\t\tAliases: []string{\"a\"},\n\t\t\t\t\t\t\tAction: withArgCheck(needAtLeast(2), withDaemon(handleRemoteFolderAdd, true)),\n\t\t\t\t\t\t}, {\n\t\t\t\t\t\t\tName: \"set\",\n\t\t\t\t\t\t\tAliases: []string{\"s\"},\n\t\t\t\t\t\t\tAction: withArgCheck(needAtLeast(2), withDaemon(handleRemoteFolderSet, true)),\n\t\t\t\t\t\t}, {\n\t\t\t\t\t\t\tName: \"remove\",\n\t\t\t\t\t\t\tAliases: []string{\"rm\"},\n\t\t\t\t\t\t\tAction: withArgCheck(needAtLeast(2), withDaemon(handleRemoteFolderRemove, true)),\n\t\t\t\t\t\t}, {\n\t\t\t\t\t\t\tName: \"clear\",\n\t\t\t\t\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleRemoteFolderClear, true)),\n\t\t\t\t\t\t}, {\n\t\t\t\t\t\t\tName: \"list\",\n\t\t\t\t\t\t\tAliases: []string{\"ls\"},\n\t\t\t\t\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleRemoteFolderList, true)),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t}, {\n\t\t\tName: \"pin\",\n\t\t\tCategory: vcscGroup,\n\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handlePin, true)),\n\t\t\tSubcommands: []cli.Command{\n\t\t\t\t{\n\t\t\t\t\tName: \"add\",\n\t\t\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handlePin, true)),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"repin\",\n\t\t\t\t\tAction: withDaemon(handleRepin, true),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"remove\",\n\t\t\t\t\tAliases: []string{\"rm\"},\n\t\t\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleUnpin, true)),\n\t\t\t\t},\n\t\t\t},\n\t\t}, {\n\t\t\tName: \"net\",\n\t\t\tCategory: netwGroup,\n\t\t\tSubcommands: []cli.Command{\n\t\t\t\t{\n\t\t\t\t\tName: \"offline\",\n\t\t\t\t\tAction: withDaemon(handleOffline, true),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"online\",\n\t\t\t\t\tAction: withDaemon(handleOnline, true),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"status\",\n\t\t\t\t\tAction: withDaemon(handleIsOnline, true),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"locate\",\n\t\t\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleNetLocate, true)),\n\t\t\t\t},\n\t\t\t},\n\t\t}, {\n\t\t\tName: \"status\",\n\t\t\tAliases: []string{\"st\"},\n\t\t\tCategory: vcscGroup,\n\t\t\tAction: withDaemon(handleStatus, true),\n\t\t}, {\n\t\t\tName: \"diff\",\n\t\t\tCategory: vcscGroup,\n\t\t\tAction: withDaemon(handleDiff, true),\n\t\t}, {\n\t\t\tName: \"tag\",\n\t\t\tCategory: vcscGroup,\n\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleTag, true)),\n\t\t}, {\n\t\t\tName: \"log\",\n\t\t\tCategory: vcscGroup,\n\t\t\tAction: withDaemon(handleLog, true),\n\t\t}, {\n\t\t\tName: \"fetch\",\n\t\t\tCategory: vcscGroup,\n\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleFetch, true)),\n\t\t}, {\n\t\t\tName: \"sync\",\n\t\t\tCategory: vcscGroup,\n\t\t\tAction: withDaemon(handleSync, true),\n\t\t}, {\n\t\t\tName: \"push\",\n\t\t\tCategory: vcscGroup,\n\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handlePush, true)),\n\t\t}, {\n\t\t\tName: \"commit\",\n\t\t\tAliases: []string{\"cmt\"},\n\t\t\tCategory: vcscGroup,\n\t\t\tAction: withDaemon(handleCommit, true),\n\t\t}, {\n\t\t\tName: \"reset\",\n\t\t\tAliases: []string{\"re\"},\n\t\t\tCategory: vcscGroup,\n\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleReset, true)),\n\t\t}, {\n\t\t\tName: \"become\",\n\t\t\tAliases: []string{\"be\"},\n\t\t\tCategory: vcscGroup,\n\t\t\tAction: withDaemon(handleBecome, true),\n\t\t}, {\n\t\t\tName: \"history\",\n\t\t\tAliases: []string{\"hst\", \"hist\"},\n\t\t\tCategory: vcscGroup,\n\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleHistory, true)),\n\t\t}, {\n\t\t\tName: \"stage\",\n\t\t\tAliases: []string{\"stg\", \"add\", \"a\"},\n\t\t\tCategory: wdirGroup,\n\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleStage, true)),\n\t\t}, {\n\t\t\tName: \"touch\",\n\t\t\tAliases: []string{\"t\"},\n\t\t\tCategory: wdirGroup,\n\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleTouch, true)),\n\t\t}, {\n\t\t\tName: \"cat\",\n\t\t\tCategory: wdirGroup,\n\t\t\tAction: withDaemon(handleCat, true),\n\t\t}, {\n\t\t\tName: \"show\",\n\t\t\tAliases: []string{\"s\", \"info\"},\n\t\t\tCategory: wdirGroup,\n\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleShow, true)),\n\t\t}, {\n\t\t\tName: \"rm\",\n\t\t\tAliases: []string{\"remove\"},\n\t\t\tCategory: wdirGroup,\n\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleRm, true)),\n\t\t}, {\n\t\t\tName: \"ls\",\n\t\t\tCategory: wdirGroup,\n\t\t\tAction: withDaemon(handleList, true),\n\t\t}, {\n\t\t\tName: \"tree\",\n\t\t\tCategory: wdirGroup,\n\t\t\tAction: withDaemon(handleTree, true),\n\t\t}, {\n\t\t\tName: \"mkdir\",\n\t\t\tCategory: wdirGroup,\n\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleMkdir, true)),\n\t\t}, {\n\t\t\tName: \"mv\",\n\t\t\tCategory: wdirGroup,\n\t\t\tAction: withArgCheck(needAtLeast(2), withDaemon(handleMv, true)),\n\t\t}, {\n\t\t\tName: \"cp\",\n\t\t\tCategory: wdirGroup,\n\t\t\tAction: withArgCheck(needAtLeast(2), withDaemon(handleCp, true)),\n\t\t}, {\n\t\t\tName: \"edit\",\n\t\t\tCategory: wdirGroup,\n\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleEdit, true)),\n\t\t}, {\n\t\t\tName: \"daemon\",\n\t\t\tCategory: repoGroup,\n\t\t\tSubcommands: []cli.Command{\n\t\t\t\t{\n\t\t\t\t\tName: \"launch\",\n\t\t\t\t\tAction: handleDaemonLaunch,\n\t\t\t\t}, {\n\t\t\t\t\tName: \"quit\",\n\t\t\t\t\tAction: withDaemon(handleDaemonQuit, false),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"ping\",\n\t\t\t\t\tAction: withDaemon(handleDaemonPing, false),\n\t\t\t\t},\n\t\t\t},\n\t\t}, {\n\t\t\tName: \"config\",\n\t\t\tAliases: []string{\"cfg\"},\n\t\t\tCategory: repoGroup,\n\t\t\tAction: withDaemon(handleConfigList, true),\n\t\t\tSubcommands: []cli.Command{\n\t\t\t\t{\n\t\t\t\t\tName: \"list\",\n\t\t\t\t\tAliases: []string{\"ls\"},\n\t\t\t\t\tAction: withDaemon(handleConfigList, true),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"get\",\n\t\t\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleConfigGet, true)),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"doc\",\n\t\t\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleConfigDoc, true)),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"set\",\n\t\t\t\t\tAction: withArgCheck(needAtLeast(2), withDaemon(handleConfigSet, true)),\n\t\t\t\t},\n\t\t\t},\n\t\t}, {\n\t\t\tName: \"fstab\",\n\t\t\tCategory: repoGroup,\n\t\t\tAction: withArgCheck(needAtLeast(0), withDaemon(handleFstabList, true)),\n\t\t\tSubcommands: []cli.Command{\n\t\t\t\t{\n\t\t\t\t\tName: \"add\",\n\t\t\t\t\tAction: withArgCheck(needAtLeast(2), withDaemon(handleFstabAdd, true)),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"remove\",\n\t\t\t\t\tAliases: []string{\"rm\"},\n\t\t\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleFstabRemove, true)),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"apply\",\n\t\t\t\t\tAction: withDaemon(handleFstabApply, true),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"list\",\n\t\t\t\t\tAliases: []string{\"ls\"},\n\t\t\t\t\tAction: withDaemon(handleFstabList, true),\n\t\t\t\t},\n\t\t\t},\n\t\t}, {\n\t\t\tName: \"trash\",\n\t\t\tAliases: []string{\"tr\"},\n\t\t\tCategory: repoGroup,\n\t\t\tAction: withDaemon(handleTrashList, true),\n\t\t\tSubcommands: []cli.Command{\n\t\t\t\t{\n\t\t\t\t\tName: \"list\",\n\t\t\t\t\tAliases: []string{\"ls\"},\n\t\t\t\t\tAction: withDaemon(handleTrashList, true),\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"undelete\",\n\t\t\t\t\tAliases: []string{\"rm\"},\n\t\t\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleTrashRemove, true)),\n\t\t\t\t},\n\t\t\t},\n\t\t}, {\n\t\t\tName: \"hints\",\n\t\t\tAliases: []string{\"hi\"},\n\t\t\tCategory: repoGroup,\n\t\t\tAction: withDaemon(handleRepoHintsList, true),\n\t\t\tSubcommands: []cli.Command{\n\t\t\t\t{\n\t\t\t\t\tName: \"list\",\n\t\t\t\t\tAliases: []string{\"ls\"},\n\t\t\t\t\tAction: withDaemon(handleRepoHintsList, true),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"set\",\n\t\t\t\t\tAliases: []string{\"s\"},\n\t\t\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleRepoHintsSet, true)),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"remove\",\n\t\t\t\t\tAliases: []string{\"rm\"},\n\t\t\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleRepoHintsRemove, true)),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"recode\",\n\t\t\t\t\tAliases: []string{\"r\"},\n\t\t\t\t\tAction: withDaemon(handleRepoHintsRecode, true),\n\t\t\t\t},\n\t\t\t},\n\t\t}, {\n\t\t\tName: \"gateway\",\n\t\t\tAliases: []string{\"gw\"},\n\t\t\tCategory: repoGroup,\n\t\t\tSubcommands: []cli.Command{\n\t\t\t\t{\n\t\t\t\t\tName: \"start\",\n\t\t\t\t\tAction: withDaemon(handleGatewayStart, true),\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"stop\",\n\t\t\t\t\tAction: withDaemon(handleGatewayStop, true),\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"status\",\n\t\t\t\t\tAction: withDaemon(handleGatewayStatus, true),\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"url\",\n\t\t\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleGatewayURL, true)),\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"user\",\n\t\t\t\t\tAliases: []string{\"u\"},\n\t\t\t\t\tSubcommands: []cli.Command{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"add\",\n\t\t\t\t\t\t\tAliases: []string{\"a\"},\n\t\t\t\t\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleGatewayUserAdd, true)),\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"remove\",\n\t\t\t\t\t\t\tAliases: []string{\"rm\"},\n\t\t\t\t\t\t\tAction: withArgCheck(needAtLeast(1), withDaemon(handleGatewayUserRemove, true)),\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"list\",\n\t\t\t\t\t\t\tAliases: []string{\"ls\"},\n\t\t\t\t\t\t\tAction: withDaemon(handleGatewayUserList, true),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t}, {\n\t\t\tName: \"debug\",\n\t\t\tAliases: []string{\"d\"},\n\t\t\tCategory: repoGroup,\n\t\t\tSubcommands: []cli.Command{\n\t\t\t\t{\n\t\t\t\t\tName: \"pprof-port\",\n\t\t\t\t\tAliases: []string{\"p\"},\n\t\t\t\t\tAction: withDaemon(handleDebugPprofPort, true),\n\t\t\t\t}, {\n\t\t\t\t\tName: \"decode-stream\",\n\t\t\t\t\tAliases: []string{\"ds\"},\n\t\t\t\t\tAction: handleDebugDecodeStream,\n\t\t\t\t}, {\n\t\t\t\t\tName: \"encode-stream\",\n\t\t\t\t\tAliases: []string{\"es\"},\n\t\t\t\t\tAction: handleDebugEncodeStream,\n\t\t\t\t}, {\n\t\t\t\t\tName: \"ten-source\",\n\t\t\t\t\tAliases: []string{\"tso\"},\n\t\t\t\t\tAction: handleDebugTenSource,\n\t\t\t\t}, {\n\t\t\t\t\tName: \"ten-sink\",\n\t\t\t\t\tAliases: []string{\"tsi\"},\n\t\t\t\t\tAction: handleDebugTenSink,\n\t\t\t\t}, {\n\t\t\t\t\tName: \"iobench\",\n\t\t\t\t\tAction: handleIOBench,\n\t\t\t\t\tSubcommands: []cli.Command{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"list\",\n\t\t\t\t\t\t\tAliases: []string{\"ls\"},\n\t\t\t\t\t\t\tAction: handleIOBenchList,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t}, {\n\t\t\t\t\tName: \"fusemock\",\n\t\t\t\t\tAction: handleDebugFuseMock,\n\t\t\t\t},\n\t\t\t},\n\t\t}, {\n\t\t\tName: \"mount\",\n\t\t\tCategory: repoGroup,\n\t\t\tAction: withDaemon(handleMount, true),\n\t\t}, {\n\t\t\tName: \"unmount\",\n\t\t\tCategory: repoGroup,\n\t\t\tAction: withDaemon(handleUnmount, true),\n\t\t}, {\n\t\t\tName: \"version\",\n\t\t\tCategory: repoGroup,\n\t\t\tAction: withDaemon(handleVersion, false),\n\t\t}, {\n\t\t\tName: \"gc\",\n\t\t\tCategory: repoGroup,\n\t\t\tAction: withDaemon(handleGc, true),\n\t\t}, {\n\t\t\tName: \"pack-repo\",\n\t\t\tCategory: repoGroup,\n\t\t\tAction: handleRepoPack,\n\t\t\tAliases: []string{\"pr\"},\n\t\t}, {\n\t\t\tName: \"unpack-repo\",\n\t\t\tCategory: repoGroup,\n\t\t\tAction: withArgCheck(needAtLeast(1), handleRepoUnpack),\n\t\t\tAliases: []string{\"ur\"},\n\t\t}, {\n\t\t\tName: \"docs\",\n\t\t\tAction: handleOpenHelp,\n\t\t\tHidden: true,\n\t\t}, {\n\t\t\tName: \"bug\",\n\t\t\tAction: handleBugReport,\n\t\t},\n\t})\n\n\texitCode := Success\n\tif err := app.Run(args); err != nil {\n\t\tlog.Error(prettyPrintError(err))\n\t\tcerr, ok := err.(ExitCode)\n\t\tif !ok {\n\t\t\texitCode = UnknownError\n\t\t}\n\n\t\texitCode = cerr.Code\n\t}\n\n\treturn exitCode\n}\n" }, { "path": "cmd/pwd/pwd-util/pwd-util.go", "content": "package main\n\nimport (\n\t\"crypto/rand\"\n\t\"fmt\"\n\n\t\"github.com/sahib/brig/cmd/pwd\"\n\t\"github.com/sahib/brig/util\"\n)\n\nfunc main() {\n\tpwd, err := pwd.PromptNewPassword(40.0)\n\tif err != nil {\n\t\tfmt.Println(\"Failed: \", err)\n\t\treturn\n\t}\n\n\tsalt := make([]byte, 32)\n\tif _, err := rand.Reader.Read(salt); err != nil {\n\t\tfmt.Println(\"Reading salt failed, you're likely doomed.\")\n\t\treturn\n\t}\n\n\tkey := util.DeriveKey([]byte(pwd), salt, 32)\n\tfmt.Printf(\"Key: %x\\nSalt: %x\\n\", key, salt)\n}\n" }, { "path": "cmd/pwd/pwd.go", "content": "package pwd\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\n\t\"github.com/chzyer/readline\"\n\t\"github.com/fatih/color\"\n\n\tzxcvbn \"github.com/nbutton23/zxcvbn-go\"\n\t\"github.com/sahib/brig/util\"\n)\n\nconst (\n\tmsgLowEntropy = \"Please enter a password with at least %g bits entropy.\"\n\tmsgReEnter = \"Well done! Please re-type your password now for safety:\"\n\tmsgBadPassword = \"This did not seem to match. Please retype it again.\"\n\tmsgMaxTriesHit = \"Maximum number of password tries exceeded: %d\"\n)\n\nfunc doPromptLine(rl *readline.Instance, prompt string, hide bool) ([]byte, error) {\n\tvar line []byte\n\tvar sline string\n\tvar err error\n\n\tif hide {\n\t\tline, err = rl.ReadPassword(prompt)\n\t} else {\n\t\tsline, err = rl.Readline()\n\t\tline = []byte(sline)\n\t}\n\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn line, nil\n}\n\nfunc createStrengthPrompt(password []rune, prefix string) string {\n\tvar symbol string\n\tvar colorFn func(format string, a ...interface{}) string\n\n\tstrength := zxcvbn.PasswordStrength(string(password), nil)\n\n\tswitch {\n\tcase strength.Entropy >= 25:\n\t\tsymbol = \"✔\"\n\t\tcolorFn = color.GreenString\n\tcase strength.Entropy >= 20:\n\t\tsymbol = \"⊞\"\n\t\tcolorFn = color.YellowString\n\tcase strength.Entropy >= 15:\n\t\tsymbol = \"⊟\"\n\t\tcolorFn = color.MagentaString\n\tdefault:\n\t\tsymbol = \"⊠\"\n\t\tcolorFn = color.RedString\n\t}\n\n\treturn colorFn(symbol + \" \" + prefix + \"passphrase: \")\n}\n\n// PromptNewPassword asks the user to input a password.\n//\n// While typing, the user gets feedback by the prompt color,\n// which changes with the security of the password to green.\n// Additionally the entrtopy of the password is shown.\n// If minEntropy was not reached after hitting enter,\n// this function will log a message and ask the user again.\nfunc PromptNewPassword(minEntropy float64) ([]byte, error) {\n\trl, err := readline.New(\"\")\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdefer util.Closer(rl)\n\n\tpasswordCfg := rl.GenPasswordConfig()\n\tpasswordCfg.SetListener(func(line []rune, pos int, key rune) (newLine []rune, newPos int, ok bool) {\n\t\trl.SetPrompt(createStrengthPrompt(line, \"New \"))\n\t\trl.Refresh()\n\t\treturn nil, 0, false\n\t})\n\n\tpwd := []byte{}\n\n\tfor {\n\t\tpwd, err = rl.ReadPasswordWithConfig(passwordCfg)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tstrength := zxcvbn.PasswordStrength(string(pwd), nil)\n\t\tif strength.Entropy >= minEntropy {\n\t\t\tbreak\n\t\t}\n\n\t\tfmt.Printf(color.YellowString(msgLowEntropy)+\"\\n\", minEntropy)\n\t}\n\n\tpasswordCfg.SetListener(func(line []rune, pos int, key rune) (newLine []rune, newPos int, ok bool) {\n\t\trl.SetPrompt(createStrengthPrompt(line, \"Retype \"))\n\t\trl.Refresh()\n\t\treturn nil, 0, false\n\t})\n\n\tfmt.Println(msgReEnter)\n\n\tfor {\n\t\tnewPwd, err := rl.ReadPasswordWithConfig(passwordCfg)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif bytes.Equal(pwd, newPwd) {\n\t\t\tbreak\n\t\t}\n\n\t\tfmt.Println(color.YellowString(msgBadPassword))\n\t}\n\n\tstrength := zxcvbn.PasswordStrength(string(pwd), nil)\n\tfmt.Printf(\n\t\t\"estimated time needed to crack password (according to zxcvbn): %s\\n\",\n\t\tcolor.BlueString(strength.CrackTimeDisplay),\n\t)\n\n\treturn pwd, nil\n}\n\nfunc promptPassword(prompt string) ([]byte, error) {\n\trl, err := readline.New(prompt)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdefer util.Closer(rl)\n\treturn doPromptLine(rl, prompt, true)\n}\n\n// PromptPassword just opens an uncolored password prompt.\n//\n// The password is not echo'd to stdout for safety reasons.\nfunc PromptPassword() ([]byte, error) {\n\treturn promptPassword(\"Password: \")\n}\n" }, { "path": "cmd/pwd/pwd_test.go", "content": "package pwd\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n\t\"time\"\n\n\tzxcvbn \"github.com/nbutton23/zxcvbn-go\"\n)\n\nfunc TestLongPassword(t *testing.T) {\n\tstart := time.Now()\n\tzxcvbn.PasswordStrength(\"68b329da9893e34099c7d8ad5cb9c940\", nil)\n\tfmt.Println(\"Took\", time.Since(start))\n}\n\nfunc BenchmarkLongPassword(b *testing.B) {\n\tfmt.Println(\"N\", b.N)\n\tfor i := 0; i < b.N; i++ {\n\t\tzxcvbn.PasswordStrength(\"1234567890123456\", nil)\n\t}\n}\n" }, { "path": "cmd/repo_handlers.go", "content": "package cmd\n\nimport (\n\t\"bytes\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"log/syslog\"\n\t\"net/url\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"runtime/trace\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/fatih/color\"\n\te \"github.com/pkg/errors\"\n\t\"github.com/sahib/brig/client\"\n\t\"github.com/sahib/brig/cmd/pwd\"\n\t\"github.com/sahib/brig/cmd/tabwriter\"\n\t\"github.com/sahib/brig/repo\"\n\t\"github.com/sahib/brig/repo/repopack\"\n\t\"github.com/sahib/brig/repo/setup\"\n\t\"github.com/sahib/brig/server\"\n\t\"github.com/sahib/brig/util\"\n\tformatter \"github.com/sahib/brig/util/log\"\n\t\"github.com/sahib/brig/version\"\n\tlog \"github.com/sirupsen/logrus\"\n\t\"github.com/urfave/cli\"\n)\n\nconst brigLogo = `\n _____ / /\\ ___ / /\\ \n / /::\\ / /::\\ / /\\ / /:/_\n / /:/\\:\\ / /:/\\:\\ / /:/ / /:/ /\\ \n / /:/~/::\\ / /:/~/:/ /__/::\\ / /:/_/::\\ \n /__/:/ /:/\\:| /__/:/ /:/___ \\__\\/\\:\\__ /__/:/__\\/\\:\\\n \\ \\:\\/:/~/:/ \\ \\:\\/:::::/ \\ \\:\\/\\ \\ \\:\\ /~~/:/\n \\ \\::/ /:/ \\ \\::/~~~~ \\__\\::/ \\ \\:\\ /:/\n \\ \\:\\/:/ \\ \\:\\ /__/:/ \\ \\:\\/:/\n \\ \\::/ \\ \\:\\ \\__\\/ \\ \\::/\n \\__\\/ \\__\\/ \\__\\/\n\n`\n\nconst initBanner = `\n\n A new file README.md was automatically added.\n Use 'brig cat README.md' to view it & get started.\n\n`\n\nfunc createInitialReadme(ctl *client.Client, folder string) error {\n\ttext := `Welcome to brig!\n\nHere's what you can do next:\n\n • Read the official documentation (Just type »brig docs«)\n • Add a few remotes to sync with (See »brig help remote«)\n • Mount your data somewhere convinient (See »brig help fstab«)\n • Sync with the remotes you've added (See »brig help sync«)\n • Have a relaxing day while exploring brig.\n\nPlease remember that brig is software in its very early stages,\nand you should not rely on it yet for production purposes.\n\nIf you're done with this README, you can easily remove it:\n\n $ brig rm README.md\n\nYour repository is here:\n\n %s\n\nHave a nice day.\n`\n\tfd, err := ioutil.TempFile(\"\", \".brig-init-readme-\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\ttext = fmt.Sprintf(text, folder)\n\tif _, err := fd.WriteString(text); err != nil {\n\t\treturn err\n\t}\n\n\treadmePath := fd.Name()\n\n\tif err := fd.Close(); err != nil {\n\t\treturn err\n\t}\n\n\tif err := ctl.Stage(readmePath, \"/README.md\"); err != nil {\n\t\treturn err\n\t}\n\n\treturn ctl.MakeCommit(\"added initial README.md\")\n}\n\nfunc isMultiAddr(ipfsPathOrMultiaddr string) bool {\n\t_, err := os.Stat(ipfsPathOrMultiaddr)\n\treturn err != nil\n}\n\nfunc handleInit(ctx *cli.Context) error {\n\tif len(ctx.Args()) == 0 {\n\t\treturn fmt.Errorf(\"Please specify a name for the owner of this repository\")\n\t}\n\n\towner := ctx.Args().First()\n\tbackend := ctx.String(\"backend\")\n\n\tfolder := ctx.String(\"repo\")\n\tif folder == \"\" {\n\t\tfolder = ctx.GlobalString(\"repo\")\n\t}\n\n\tif ctx.NArg() == 2 {\n\t\tvar err error\n\t\tfolder, err = filepath.Abs(ctx.Args().Get(1))\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to get absolute path for %s: %v\", folder, err)\n\t\t}\n\t}\n\n\tif ctx.NArg() > 2 {\n\t\treturn fmt.Errorf(\"too many arguments\")\n\t}\n\n\tif folder == \"\" {\n\t\tvar err error\n\t\tfolder, err = guessRepoFolder(ctx)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfmt.Printf(\"-- Guessed folder for init: %s\\n\", folder)\n\t}\n\n\t// doing init twice can easily break things.\n\tisInitialized, err := isNonEmptyDir(folder)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif isInitialized {\n\t\treturn fmt.Errorf(\"`%s` already exists and is not empty; refusing to do init\", folder)\n\t}\n\n\tipfsPathOrMultiaddr := ctx.String(\"ipfs-path-or-multiaddr\")\n\tdoIpfsSetup := !ctx.Bool(\"no-ipfs-setup\")\n\tdoIpfsConfig := !ctx.Bool(\"no-ipfs-config\")\n\tdoExtraIpfsConfig := !ctx.Bool(\"no-ipfs-optimization\")\n\n\tipfsRepoPath := ipfsPathOrMultiaddr\n\tisMa := isMultiAddr(ipfsPathOrMultiaddr)\n\tif isMa {\n\t\t// NOTE: If we're connecting over a multiaddr,\n\t\t// then we should not setup an ipfs repo.\n\t\t// Assumption is that it exists already.\n\t\tdoIpfsSetup = false\n\t\tipfsRepoPath = \"\"\n\t}\n\n\tif backend == \"httpipfs\" {\n\t\tif _, err := setup.IPFS(setup.Options{\n\t\t\tLogWriter: os.Stdout,\n\t\t\tSetup: doIpfsSetup,\n\t\t\tSetDefaultConfig: doIpfsConfig,\n\t\t\tSetExtraConfig: doExtraIpfsConfig,\n\t\t\tIpfsPath: ipfsRepoPath,\n\t\t}); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tdaemonURL, err := guessFreeDaemonURL(ctx, owner)\n\tif err != nil {\n\t\tlog.WithError(err).Warnf(\"failed to figure out a free daemon url\")\n\t}\n\n\tif err := Init(\n\t\tctx,\n\t\tipfsPathOrMultiaddr,\n\t\trepo.InitOptions{\n\t\t\tBaseFolder: folder,\n\t\t\tOwner: owner,\n\t\t\tBackendName: backend,\n\t\t\tDaemonURL: daemonURL,\n\t\t},\n\t); err != nil {\n\t\treturn ExitCode{UnknownError, fmt.Sprintf(\"init failed: %v\", err)}\n\t}\n\n\t// Start the daemon on the freshly initialized repo:\n\tctl, err := startDaemon(ctx, folder, daemonURL)\n\tif err != nil {\n\t\treturn ExitCode{\n\t\t\tDaemonNotResponding,\n\t\t\tfmt.Sprintf(\"Unable to start daemon: %v\", err),\n\t\t}\n\t}\n\n\t// Run the actual handler:\n\tdefer ctl.Close()\n\n\treturn handleInitPost(ctx, ctl, folder)\n}\n\nfunc handleInitPost(ctx *cli.Context, ctl *client.Client, folder string) error {\n\tif !ctx.Bool(\"empty\") {\n\t\tif err := createInitialReadme(ctl, folder); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tif !ctx.Bool(\"no-logo\") {\n\t\tfmt.Println(brigLogo)\n\n\t\tif !ctx.Bool(\"empty\") {\n\t\t\tfmt.Println(initBanner)\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc printConfigDocEntry(entry client.ConfigEntry) {\n\tval := entry.Val\n\tif val == \"\" {\n\t\tval = color.YellowString(\"(empty)\")\n\t}\n\n\tdefaultMarker := \"\"\n\tif entry.Val == entry.Default {\n\t\tdefaultMarker = color.CyanString(\"(default)\")\n\t}\n\n\tfmt.Printf(\"%s: %v %s\\n\", color.GreenString(entry.Key), val, defaultMarker)\n\n\tneedsRestart := yesify(entry.NeedsRestart)\n\tdefaultVal := entry.Default\n\tif entry.Default == \"\" {\n\t\tdefaultVal = color.YellowString(\"(empty)\")\n\t}\n\n\tfmt.Printf(\" Default: %v\\n\", defaultVal)\n\tfmt.Printf(\" Documentation: %v\\n\", entry.Doc)\n\tfmt.Printf(\" Needs restart: %v\\n\", needsRestart)\n}\n\nfunc handleConfigList(cli *cli.Context, ctl *client.Client) error {\n\tall, err := ctl.ConfigAll()\n\tif err != nil {\n\t\treturn ExitCode{UnknownError, fmt.Sprintf(\"config list: %v\", err)}\n\t}\n\n\tfor _, entry := range all {\n\t\tprintConfigDocEntry(entry)\n\t}\n\n\treturn nil\n}\n\nfunc handleConfigGet(ctx *cli.Context, ctl *client.Client) error {\n\tkey := ctx.Args().Get(0)\n\tval, err := ctl.ConfigGet(key)\n\tif err != nil {\n\t\treturn ExitCode{UnknownError, fmt.Sprintf(\"config get: %v\", err)}\n\t}\n\n\tfor _, elem := range strings.Split(val, \" ;; \") {\n\t\tfmt.Println(elem)\n\t}\n\treturn nil\n}\n\nfunc handleConfigSet(ctx *cli.Context, ctl *client.Client) error {\n\tkey := ctx.Args().Get(0)\n\n\tval := ctx.Args().Get(1)\n\tif len(ctx.Args()) > 2 {\n\t\tval = strings.Join(ctx.Args()[1:], \" ;; \")\n\t}\n\n\tif err := ctl.ConfigSet(key, val); err != nil {\n\t\treturn ExitCode{UnknownError, fmt.Sprintf(\"config set: %v\", err)}\n\t}\n\n\tentry, err := ctl.ConfigDoc(key)\n\tif err != nil {\n\t\treturn ExitCode{UnknownError, fmt.Sprintf(\"config doc: %v\", err)}\n\t}\n\n\tif entry.NeedsRestart {\n\t\tfmt.Println(\"NOTE: You need to restart brig for this option to take effect.\")\n\t}\n\n\treturn nil\n}\n\nfunc handleConfigDoc(ctx *cli.Context, ctl *client.Client) error {\n\tkey := ctx.Args().Get(0)\n\tentry, err := ctl.ConfigDoc(key)\n\tif err != nil {\n\t\treturn ExitCode{UnknownError, fmt.Sprintf(\"config get: %v\", err)}\n\t}\n\n\tprintConfigDocEntry(entry)\n\treturn nil\n}\n\nfunc handleDaemonPing(ctx *cli.Context, ctl *client.Client) error {\n\tcount := ctx.Int(\"count\")\n\tfor i := 0; i < count; i++ {\n\t\tbefore := time.Now()\n\t\tsymbol := color.GreenString(\"✔\")\n\n\t\tif err := ctl.Ping(); err != nil {\n\t\t\tsymbol = color.RedString(\"✘\")\n\t\t}\n\n\t\tdelay := time.Since(before)\n\t\tfmt.Printf(\"#%02d %s ➔ %s: %s (%v)\\n\",\n\t\t\ti+1,\n\t\t\tctl.LocalAddr().String(),\n\t\t\tctl.RemoteAddr().String(),\n\t\t\tsymbol,\n\t\t\tdelay,\n\t\t)\n\n\t\ttime.Sleep(1 * time.Second)\n\t}\n\n\treturn nil\n}\n\nfunc handleDaemonQuit(ctx *cli.Context, ctl *client.Client) error {\n\tif err := ctl.Quit(); err != nil {\n\t\treturn ExitCode{\n\t\t\tDaemonNotResponding,\n\t\t\tfmt.Sprintf(\"brigd not responding: %v\", err),\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc switchToSyslog() {\n\twSyslog, err := syslog.New(syslog.LOG_NOTICE, \"brig\")\n\tif err != nil {\n\t\tlog.Warningf(\"failed to open connection to syslog for brig: %v\", err)\n\t\tlogFd, err := ioutil.TempFile(\"\", \"brig-*.log\")\n\t\tif err != nil {\n\t\t\tlog.Warningf(\"\")\n\t\t} else {\n\t\t\tlog.Warningf(\"Will log to %s from now on.\", logFd.Name())\n\t\t\tlog.SetOutput(logFd)\n\t\t}\n\n\t\treturn\n\t}\n\n\tlog.SetLevel(log.DebugLevel)\n\tlog.SetFormatter(&formatter.FancyLogFormatter{\n\t\tUseColors: false,\n\t})\n\n\t// TODO: we should also forward panics (os.Stderr) to syslog.\n\t// They don't come from the log obviously though.\n\tlog.SetOutput(\n\t\tio.MultiWriter(\n\t\t\tformatter.NewSyslogWrapper(wSyslog),\n\t\t\tos.Stdout,\n\t\t),\n\t)\n}\n\nfunc handleDaemonLaunch(ctx *cli.Context) error {\n\t// Enable tracing (for profiling) if required.\n\tif ctx.Bool(\"trace\") {\n\t\ttracePath := fmt.Sprintf(\"/tmp/brig-%d.trace\", os.Getpid())\n\t\tlog.Debugf(\"Writing trace output to %s\", tracePath)\n\t\tfd, err := os.Create(tracePath)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tdefer util.Closer(fd)\n\n\t\tif err := trace.Start(fd); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tdefer trace.Stop()\n\t}\n\n\trepoPath, err := guessRepoFolder(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdaemonURL, err := guessDaemonURL(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Make sure IPFS is running. Also set required options,\n\t// but don't bother to set optimizations.\n\tvar ipfsPath string\n\tcfg, err := openConfig(repoPath)\n\tif err != nil {\n\t\tlog.Warningf(\"failed to read config at %v: %v\", repoPath, err)\n\t} else {\n\t\tipfsPath = cfg.String(\"daemon.ipfs_path_or_url\")\n\t}\n\n\tif _, err := setup.IPFS(setup.Options{\n\t\tLogWriter: &logWriter{prefix: \"ipfs\"},\n\t\tSetup: true,\n\t\tSetDefaultConfig: true,\n\t\tSetExtraConfig: false,\n\t\tIpfsPath: ipfsPath,\n\t}); err != nil {\n\t\treturn err\n\t}\n\n\tlogToStdout := ctx.Bool(\"log-to-stdout\")\n\tif !logToStdout {\n\t\tlog.Infof(\"all further logs will be also piped to the syslog daemon.\")\n\t\tlog.Infof(\"Use »journalctl -fet brig« to view logs.\")\n\t\tswitchToSyslog()\n\t} else {\n\t\tlog.SetOutput(os.Stdout)\n\t}\n\n\tserver, err := server.BootServer(repoPath, daemonURL)\n\tif err != nil {\n\t\treturn ExitCode{\n\t\t\tUnknownError,\n\t\t\tfmt.Sprintf(\"failed to boot brigd: %v\", err),\n\t\t}\n\t}\n\n\tdefer util.Closer(server)\n\n\tif err := server.Serve(); err != nil {\n\t\treturn ExitCode{\n\t\t\tUnknownError,\n\t\t\tfmt.Sprintf(\"failed to serve: %v\", err),\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc handleMount(ctx *cli.Context, ctl *client.Client) error {\n\tmountPath := ctx.Args().First()\n\tabsMountPath, err := filepath.Abs(mountPath)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif !ctx.Bool(\"no-mkdir\") {\n\t\tif _, err := os.Stat(absMountPath); os.IsNotExist(err) {\n\t\t\tfmt.Printf(\n\t\t\t\t\"Mount directory »%s« does not exist. Will create it.\\n\",\n\t\t\t\tabsMountPath,\n\t\t\t)\n\t\t\tif err := os.MkdirAll(absMountPath, 0700); err != nil {\n\t\t\t\treturn e.Wrapf(err, \"failed to mkdir mount point\")\n\t\t\t}\n\t\t}\n\t}\n\n\toptions := client.MountOptions{\n\t\tReadOnly: ctx.Bool(\"readonly\"),\n\t\tOffline: ctx.Bool(\"offline\"),\n\t\tRootPath: ctx.String(\"root\"),\n\t}\n\n\tif err := ctl.Mount(absMountPath, options); err != nil {\n\t\treturn ExitCode{\n\t\t\tUnknownError,\n\t\t\tfmt.Sprintf(\"Failed to mount: %v\", err),\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc handleUnmount(ctx *cli.Context, ctl *client.Client) error {\n\tmountPath := ctx.Args().First()\n\tabsMountPath, err := filepath.Abs(mountPath)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif err := ctl.Unmount(absMountPath); err != nil {\n\t\treturn ExitCode{\n\t\t\tUnknownError,\n\t\t\tfmt.Sprintf(\"Failed to unmount: %v\", err),\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc handleVersion(ctx *cli.Context, ctl *client.Client) error {\n\tvInfo, err := ctl.Version()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\trow := func(name, value string) {\n\t\tfmt.Printf(\"%25s: %s\\n\", name, value)\n\t}\n\n\trow(\"Client Version\", version.String())\n\trow(\"Client Rev\", version.GitRev)\n\trow(\"Server Version\", vInfo.ServerSemVer)\n\trow(\"Server Rev\", vInfo.ServerRev)\n\trow(\"Backend (ipfs) Version\", vInfo.BackendSemVer)\n\trow(\"Backend (ipfs) Rev\", vInfo.BackendRev)\n\trow(\"Build time\", version.BuildTime)\n\n\treturn nil\n}\n\nfunc handleGc(ctx *cli.Context, ctl *client.Client) error {\n\taggressive := ctx.Bool(\"aggressive\")\n\tfreed, err := ctl.GarbageCollect(aggressive)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif len(freed) == 0 {\n\t\tfmt.Println(\"Nothing freed.\")\n\t\treturn nil\n\t}\n\n\ttabW := tabwriter.NewWriter(\n\t\tos.Stdout, 0, 0, 2, ' ',\n\t\ttabwriter.StripEscape,\n\t)\n\n\tfmt.Fprintln(tabW, \"CONTENT\\tHASH\\tOWNER\\t\")\n\n\tfor _, gcItem := range freed {\n\t\tfmt.Fprintf(\n\t\t\ttabW,\n\t\t\t\"%s\\t%s\\t%s\\t\\n\",\n\t\t\tcolor.WhiteString(gcItem.Path),\n\t\t\tcolor.RedString(gcItem.Content.ShortB58()),\n\t\t\tcolor.CyanString(gcItem.Owner),\n\t\t)\n\t}\n\n\treturn tabW.Flush()\n}\n\nfunc handleFstabAdd(ctx *cli.Context, ctl *client.Client) error {\n\tmountName := ctx.Args().Get(0)\n\tmountPath := ctx.Args().Get(1)\n\n\toptions := client.MountOptions{\n\t\tReadOnly: ctx.Bool(\"readonly\"),\n\t\tRootPath: ctx.String(\"root\"),\n\t\tOffline: ctx.Bool(\"offline\"),\n\t}\n\n\treturn ctl.FstabAdd(mountName, mountPath, options)\n}\n\nfunc handleFstabRemove(ctx *cli.Context, ctl *client.Client) error {\n\tmountName := ctx.Args().Get(0)\n\treturn ctl.FstabRemove(mountName)\n}\n\nfunc handleFstabApply(ctx *cli.Context, ctl *client.Client) error {\n\tif ctx.Bool(\"unmount\") {\n\t\treturn ctl.FstabUnmountAll()\n\t}\n\n\treturn ctl.FstabApply()\n}\n\nfunc handleFstabUnmounetAll(ctx *cli.Context, ctl *client.Client) error {\n\treturn ctl.FstabUnmountAll()\n}\n\nfunc handleFstabList(ctx *cli.Context, ctl *client.Client) error {\n\tmounts, err := ctl.FsTabList()\n\tif err != nil {\n\t\treturn ExitCode{UnknownError, fmt.Sprintf(\"config list: %v\", err)}\n\t}\n\n\tif len(mounts) == 0 {\n\t\treturn nil\n\t}\n\n\ttabW := tabwriter.NewWriter(\n\t\tos.Stdout, 0, 0, 2, ' ',\n\t\ttabwriter.StripEscape,\n\t)\n\n\ttmpl, err := readFormatTemplate(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif tmpl == nil && len(mounts) != 0 {\n\t\tfmt.Fprintln(tabW, \"NAME\\tPATH\\tREAD_ONLY\\tOFFLINE\\tROOT\\tACTIVE\\t\")\n\t}\n\n\tfor _, entry := range mounts {\n\t\tif tmpl != nil {\n\t\t\tif err := tmpl.Execute(os.Stdout, entry); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tcontinue\n\t\t}\n\n\t\tfmt.Fprintf(\n\t\t\ttabW,\n\t\t\t\"%s\\t%s\\t%s\\t%s\\t%s\\t%s\\n\",\n\t\t\tentry.Name,\n\t\t\tentry.Path,\n\t\t\tyesify(entry.ReadOnly),\n\t\t\tyesify(entry.Offline),\n\t\t\tentry.Root,\n\t\t\tcheckmarkify(entry.Active),\n\t\t)\n\t}\n\n\treturn tabW.Flush()\n}\n\nfunc handleGatewayStart(ctx *cli.Context, ctl *client.Client) error {\n\tisEnabled, err := ctl.ConfigGet(\"gateway.enabled\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif isEnabled == \"false\" {\n\t\tif err := ctl.ConfigSet(\"gateway.enabled\", \"true\"); err != nil {\n\t\t\treturn err\n\t\t}\n\t} else {\n\t\tfmt.Println(\"Seems like we're running already.\")\n\t}\n\n\tport, err := ctl.ConfigGet(\"gateway.port\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdomain := \"localhost\"\n\tprotocol := \"http\"\n\turl := fmt.Sprintf(\"%s://%s:%s\", protocol, domain, port)\n\tfmt.Printf(\"The gateway is accessible via %s\\n\", url)\n\treturn nil\n}\n\nfunc handleGatewayStatus(ctx *cli.Context, ctl *client.Client) error {\n\tisEnabled, err := ctl.ConfigGet(\"gateway.enabled\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif isEnabled == \"false\" {\n\t\tfmt.Println(\"• The gateway is not running. Use »brig gateway start« to start.\")\n\t\treturn nil\n\t}\n\n\tport, err := ctl.ConfigGet(\"gateway.port\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdomain := \"localhost\"\n\tprotocol := \"http\"\n\turl := fmt.Sprintf(\"%s://%s:%s\", protocol, domain, port)\n\n\tfmt.Printf(\"• Running on %s\\n\", color.GreenString(url))\n\tuiIsEnabled, err := ctl.ConfigGet(\"gateway.ui.enabled\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif uiIsEnabled == \"true\" {\n\t\tfmt.Println(\"• The Web UI is currently enabled and can be accessed via the URL above.\")\n\t\tfmt.Println(\" If you want to disable the UI (»/get« will still work), then do:\")\n\t\tfmt.Println(\"\")\n\t\tfmt.Println(\" $ brig cfg gateway.ui.enabled false\")\n\t\tfmt.Println(\"\")\n\t} else {\n\t\tfmt.Println(\"• There is no UI enabled. You can enable it via:\")\n\t\tfmt.Println(\"\")\n\t\tfmt.Println(\" $ brig cfg gateway.ui.enabled true\")\n\t\tfmt.Println(\"\")\n\t}\n\n\tusers, err := ctl.GatewayUserList()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tauthIsEnabled := len(users) > 0\n\tif authIsEnabled {\n\t\tfmt.Printf(\n\t\t\t\"• There are %s users currently. Review them with »brig gw user ls«.\\n\",\n\t\t\tcolor.GreenString(fmt.Sprintf(\"%d\", len(users))),\n\t\t)\n\t} else {\n\t\tfmt.Printf(\"• There is %s user authentication enabled.\\n\", color.YellowString(\"no\"))\n\t\tfmt.Printf(\" You can enable it by setting the following config keys:\\n\")\n\t\tfmt.Printf(\"\\n\")\n\t\tfmt.Printf(\" $ brig gateway user add --role-admin \\n\")\n\t\tfmt.Printf(\"\\n\")\n\t}\n\n\treturn nil\n}\n\nfunc handleGatewayStop(ctx *cli.Context, ctl *client.Client) error {\n\tisEnabled, err := ctl.ConfigGet(\"gateway.enabled\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif isEnabled == \"true\" {\n\t\tif err := ctl.ConfigSet(\"gateway.enabled\", \"false\"); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfmt.Println(\"The gateway will stop serving after handling all open requests.\")\n\t} else {\n\t\tfmt.Println(\"It seems like the gateway is already stopped.\")\n\t}\n\n\treturn nil\n}\n\nfunc handleGatewayURL(ctx *cli.Context, ctl *client.Client) error {\n\tpath := ctx.Args().First()\n\tif _, err := ctl.Stat(path); err != nil {\n\t\treturn err\n\t}\n\n\tdomain := \"localhost\"\n\tport, err := ctl.ConfigGet(\"gateway.port\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif port == \"80\" || port == \"443\" {\n\t\tport = \"\"\n\t} else {\n\t\tport = \":\" + port\n\t}\n\n\tprotocol := \"http\"\n\tescapedPath := url.PathEscape(strings.TrimLeft(path, \"/\"))\n\tfmt.Printf(\"%s://%s%s/get/%s\\n\", protocol, domain, port, escapedPath)\n\treturn nil\n}\n\nfunc handleGatewayUserAdd(ctx *cli.Context, ctl *client.Client) error {\n\tnArgs := len(ctx.Args())\n\tname := ctx.Args().First()\n\n\tvar password string\n\tif nArgs > 1 {\n\t\tpassword = ctx.Args().Get(1)\n\t} else {\n\t\tbPassword, err := pwd.PromptNewPassword(14)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tpassword = string(bPassword)\n\t}\n\n\tfolders := []string{\"/\"}\n\tif nArgs > 2 {\n\t\tfolders = ctx.Args()[2:]\n\t}\n\n\tallRights := []string{\n\t\t\"fs.download\",\n\t\t\"fs.view\",\n\t\t\"fs.edit\",\n\t\t\"remotes.view\",\n\t\t\"remotes.edit\",\n\t}\n\n\trights := []string{}\n\tif ctx.Bool(\"role-admin\") {\n\t\trights = allRights\n\t}\n\n\tif ctx.Bool(\"role-editor\") {\n\t\trights = allRights[:len(allRights)-1]\n\t}\n\n\tif ctx.Bool(\"role-collaborator\") {\n\t\trights = allRights[:len(allRights)-2]\n\t}\n\n\tif ctx.Bool(\"role-viewer\") {\n\t\trights = allRights[:len(allRights)-3]\n\t}\n\n\tif ctx.Bool(\"role-link-only\") {\n\t\trights = allRights[:len(allRights)-4]\n\t}\n\n\tif r := ctx.String(\"rights\"); r != \"\" {\n\t\trights = strings.Split(r, \",\")\n\t}\n\n\treturn ctl.GatewayUserAdd(name, password, folders, rights)\n}\n\nfunc handleGatewayUserRemove(ctx *cli.Context, ctl *client.Client) error {\n\tfor _, name := range ctx.Args() {\n\t\tif err := ctl.GatewayUserRemove(name); err != nil {\n\t\t\tfmt.Printf(\"Failed to remove »%s«: %v\\n\", name, err)\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc handleGatewayUserList(ctx *cli.Context, ctl *client.Client) error {\n\tusers, err := ctl.GatewayUserList()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\ttabW := tabwriter.NewWriter(\n\t\tos.Stdout, 0, 0, 2, ' ',\n\t\ttabwriter.StripEscape,\n\t)\n\n\ttmpl, err := readFormatTemplate(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif tmpl == nil {\n\t\tif len(users) == 0 {\n\t\t\tfmt.Println(\"No users. Add some with »brig gw user add «\")\n\t\t} else {\n\t\t\tfmt.Fprintln(tabW, \"NAME\\tFOLDERS\\tRIGHTS\\t\")\n\t\t}\n\t}\n\n\tfor _, user := range users {\n\t\tif tmpl != nil {\n\t\t\tif err := tmpl.Execute(os.Stdout, user); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tcontinue\n\t\t}\n\n\t\tfmt.Fprintf(\n\t\t\ttabW,\n\t\t\t\"%s\\t%s\\t%s\\t\\n\",\n\t\t\tuser.Name,\n\t\t\tstrings.Join(user.Folders, \",\"),\n\t\t\tstrings.Join(user.Rights, \",\"),\n\t\t)\n\t}\n\n\treturn tabW.Flush()\n}\n\nfunc readPassword(ctx *cli.Context, isNew bool) ([]byte, error) {\n\tif ctx.IsSet(\"password-command\") {\n\t\tlog.Debugf(\"reading by password command.\")\n\t\tcmd := exec.Command(\"/bin/sh\", \"-c\", ctx.String(\"password-command\"))\n\n\t\t// Make sure sub command can access our streams.\n\t\t// Some password managers might ask for a master password.\n\t\tcmd.Stderr = os.Stderr\n\t\tcmd.Stdin = os.Stdin\n\n\t\tout, err := cmd.Output()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\t// Strip any newline produced by the tool.\n\t\t// Just hope that nobody really tries to use newlines\n\t\t// as part of the password. Would still work though\n\t\t// as long only --password-command is used to enter the password.\n\t\treturn bytes.TrimRight(out, \"\\n\\r\"), nil\n\t}\n\n\tif ctx.IsSet(\"password-file\") {\n\t\tlog.Debugf(\"reading from password file.\")\n\t\treturn ioutil.ReadFile(ctx.String(\"password-file\"))\n\t}\n\n\tif isNew {\n\t\treturn pwd.PromptNewPassword(10)\n\t}\n\n\treturn pwd.PromptPassword()\n}\n\nfunc handleRepoPack(ctx *cli.Context) error {\n\tfolder, err := guessRepoFolder(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tisRunning, err := isDaemonRunning(ctx)\n\tif err != nil {\n\t\treturn e.Wrap(err, \"failed to check if daemon is running\")\n\t}\n\n\tif isRunning {\n\t\tlog.Error(\"daemon is still running for this repo, please quit it first!\")\n\t\tlog.Errorf(\"Use »brig --repo %s daemon quit« for this.\", folder)\n\t\treturn errors.New(\"refusing to pack data, there might be inconsistencies\")\n\t}\n\n\tpass, err := readPassword(ctx, true)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tarchivePath := ctx.Args().First()\n\tif archivePath == \"\" {\n\t\tarchivePath = folder + \".repopack\"\n\t}\n\n\tlog.Infof(\"writing archive to »%s«\", archivePath)\n\treturn repopack.PackRepo(\n\t\tfolder,\n\t\tarchivePath,\n\t\tstring(pass),\n\t\t!ctx.Bool(\"no-remove\"),\n\t)\n}\n\nfunc handleRepoUnpack(ctx *cli.Context) error {\n\tarchivePath := ctx.Args().First()\n\tfolder, err := guessRepoFolder(ctx)\n\tif err != nil {\n\t\t// Small convenience hack: if the archive ends in .repopack\n\t\t// assume that it was created from a repo with the same path\n\t\t// but without the suffix.\n\t\tfolder = strings.TrimSuffix(archivePath, \".repopack\")\n\t}\n\n\tisNonEmpty, err := isNonEmptyDir(folder)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif isNonEmpty {\n\t\treturn fmt.Errorf(\"»%s« is non-empty, refusing to overwrite\", folder)\n\t}\n\n\tif archivePath == \"\" {\n\t\treturn fmt.Errorf(\"please specify the location of the packed archive\")\n\t}\n\n\tpass, err := readPassword(ctx, false)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tlog.Infof(\"unpacking to »%s«\", folder)\n\treturn repopack.UnpackRepo(\n\t\tfolder,\n\t\tarchivePath,\n\t\tstring(pass),\n\t\t!ctx.Bool(\"no-remove\"),\n\t)\n}\n\nfunc optionalStringParamAsPtr(ctx *cli.Context, name string) *string {\n\tif v := ctx.String(name); v != \"\" {\n\t\treturn &v\n\t}\n\n\treturn nil\n}\n\nfunc handleRepoHintsSet(ctx *cli.Context, ctl *client.Client) error {\n\tpath := ctx.Args().First()\n\n\tif !ctx.Bool(\"force\") {\n\t\tif _, err := ctl.Stat(path); err != nil {\n\t\t\treturn fmt.Errorf(\"no file or directory at »%s« (use --force to create anyways)\", path)\n\t\t}\n\t}\n\n\tzipHint := optionalStringParamAsPtr(ctx, \"compression\")\n\tencHint := optionalStringParamAsPtr(ctx, \"encryption\")\n\n\t// TODO: There seems to be a bug in the cli library.\n\t// When --recode comes directly after 'set' then\n\t// all other arguments are part of 'ctx.Args()' and do not get\n\t// parsed. This check at least catches this behavior.\n\tif zipHint == nil && encHint == nil {\n\t\treturn fmt.Errorf(\"need at least one of --encryption or --compression\")\n\t}\n\n\tif err := ctl.HintSet(path, zipHint, encHint); err != nil {\n\t\treturn err\n\t}\n\n\tif ctx.Bool(\"recode\") {\n\t\treturn ctl.RecodeStream(path)\n\t}\n\n\treturn nil\n}\n\nfunc handleRepoHintsList(ctx *cli.Context, ctl *client.Client) error {\n\thints, err := ctl.HintList()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif len(ctx.Args()) != 0 {\n\t\treturn fmt.Errorf(\"extra arguments passed\")\n\t}\n\n\ttabW := tabwriter.NewWriter(\n\t\tos.Stdout, 0, 0, 2, ' ',\n\t\ttabwriter.StripEscape,\n\t)\n\n\tfmt.Fprintln(tabW, \"PATH\\tENCRYPTION\\tCOMPRESSION\\t\")\n\n\tfor _, hint := range hints {\n\t\tfmt.Fprintf(\n\t\t\ttabW,\n\t\t\t\"%s\\t%s\\t%s\\t\\n\",\n\t\t\thint.Path,\n\t\t\thint.EncryptionAlgo,\n\t\t\thint.CompressionAlgo,\n\t\t)\n\t}\n\n\treturn tabW.Flush()\n}\n\nfunc handleRepoHintsRemove(ctx *cli.Context, ctl *client.Client) error {\n\treturn ctl.HintRemove(ctx.Args().First())\n}\n\nfunc handleRepoHintsRecode(ctx *cli.Context, ctl *client.Client) error {\n\trepoPath := ctx.Args().Get(0)\n\tif repoPath == \"\" {\n\t\trepoPath = \"/\"\n\n\t}\n\n\treturn ctl.RecodeStream(repoPath)\n}\n" }, { "path": "cmd/suggest.go", "content": "package cmd\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"sort\"\n\t\"strings\"\n\n\t\"github.com/fatih/color\"\n\t\"github.com/sahib/brig/client\"\n\t\"github.com/urfave/cli\"\n\t\"github.com/xrash/smetrics\"\n)\n\ntype suggestion struct {\n\tname string\n\tscore float64\n}\n\nfunc levenshteinRatio(s, t string) float64 {\n\tlensum := float64(len(s) + len(t))\n\tif lensum == 0 {\n\t\treturn 1.0\n\t}\n\n\tdist := float64(smetrics.WagnerFischer(s, t, 1, 1, 2))\n\treturn (lensum - dist) / lensum\n}\n\nfunc findLastGoodCommands(ctx *cli.Context) ([]string, []cli.Command) {\n\tfor ctx.Parent() != nil {\n\t\tctx = ctx.Parent()\n\t}\n\n\targs := ctx.Args()\n\tif len(args) == 0 || len(args) == 1 {\n\t\treturn nil, ctx.App.Commands\n\t}\n\n\tcmd := ctx.App.Command(args[0])\n\tif cmd == nil {\n\t\treturn nil, ctx.App.Commands\n\t}\n\n\tvalidArgs := []string{args[0]}\n\targs = args[1 : len(args)-1]\n\n\tfor len(args) != 0 && cmd != nil {\n\t\tfor _, subCmd := range cmd.Subcommands {\n\t\t\tif subCmd.Name == args[0] {\n\t\t\t\tcmd = &subCmd\n\t\t\t}\n\t\t}\n\n\t\tvalidArgs = append(validArgs, args[0])\n\t\targs = args[1:]\n\t}\n\n\treturn validArgs, cmd.Subcommands\n}\n\nfunc findSimilarCommands(cmdName string, cmds []cli.Command) []suggestion {\n\tsimilars := []suggestion{}\n\n\tfor _, cmd := range cmds {\n\t\tcandidates := []string{cmd.Name}\n\t\tcandidates = append(candidates, cmd.Aliases...)\n\n\t\tfor _, candidate := range candidates {\n\t\t\tif score := levenshteinRatio(cmdName, candidate); score >= 0.6 {\n\t\t\t\tsimilars = append(similars, suggestion{\n\t\t\t\t\tname: cmd.Name,\n\t\t\t\t\tscore: score,\n\t\t\t\t})\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t}\n\n\t// Special cases for the git inclined:\n\tstaticSuggestions := map[string]string{\n\t\t\"insert\": \"stage\",\n\t\t\"pull\": \"sync\",\n\t\t\"merge\": \"sync\",\n\t}\n\n\tfor gitName, brigName := range staticSuggestions {\n\t\tif cmdName == gitName {\n\t\t\tsimilars = append(similars, suggestion{\n\t\t\t\tname: brigName,\n\t\t\t\tscore: 0.0,\n\t\t\t})\n\t\t}\n\t}\n\n\t// Let suggestions be sorted by their similarity:\n\tsort.Slice(similars, func(i, j int) bool {\n\t\treturn similars[i].score < similars[j].score\n\t})\n\n\treturn similars\n}\n\nfunc findCurrentCommand(ctx *cli.Context) *cli.Command {\n\tfor {\n\t\tpar := ctx.Parent()\n\t\tif par == nil {\n\t\t\tbreak\n\t\t}\n\n\t\tctx = par\n\t}\n\n\tvar command *cli.Command\n\tfor args := ctx.Args(); len(args) > 0; {\n\t\tsubCommand := ctx.App.Command(args[0])\n\t\targs = args[1:]\n\t\tif subCommand != nil {\n\t\t\tcommand = subCommand\n\t\t}\n\t}\n\n\treturn command\n}\n\nfunc completeLocalPath(ctx *cli.Context) {\n\tfiles, err := ioutil.ReadDir(\"./\")\n\tif err != nil {\n\t\treturn\n\t}\n\n\tfor _, file := range files {\n\t\tfmt.Println(file.Name())\n\t}\n}\n\nfunc completeBrigPath(allowFiles, allowDirs bool) func(ctx *cli.Context) {\n\treturn func(ctx *cli.Context) {\n\t\t// Check if the daemon is running:\n\t\tdaemonURL, err := guessDaemonURL(ctx)\n\t\tif err != nil {\n\t\t\treturn\n\t\t}\n\n\t\tctl, err := client.Dial(context.Background(), daemonURL)\n\t\tif err != nil {\n\t\t\treturn\n\t\t}\n\n\t\tstats, err := ctl.List(\"/\", -1)\n\t\tif err != nil {\n\t\t\treturn\n\t\t}\n\n\t\tfor _, stat := range stats {\n\t\t\tif stat.Path == \"/\" {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tif stat.IsDir && allowDirs {\n\t\t\t\tfmt.Println(stat.Path)\n\t\t\t}\n\n\t\t\tif !stat.IsDir && allowFiles {\n\t\t\t\tfmt.Println(stat.Path)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc completeArgsUsage(ctx *cli.Context) {\n\tif command := findCurrentCommand(ctx); command != nil {\n\t\tif len(command.Flags) == 0 {\n\t\t\treturn\n\t\t}\n\n\t\tfor _, flag := range command.Flags {\n\t\t\tsplit := strings.SplitN(flag.GetName(), \",\", 2)\n\t\t\tlongName := split[0]\n\t\t\tfmt.Printf(\"--%s\\n\", longName)\n\t\t}\n\n\t\tfmt.Println(command.ArgsUsage)\n\t}\n}\n\nfunc completeLocalFile(ctx *cli.Context) {\n\tif len(os.Args) >= 2 {\n\t\tlastArg := os.Args[len(os.Args)-2]\n\t\tcmd := findCurrentCommand(ctx)\n\t\tif lastArg != cmd.FullName() {\n\t\t\treturn\n\t\t}\n\t}\n\n\t// CAVEAT: We currently do not get partial words from bash/zsh.\n\t// e.g. \"brig stage /us\" will pass the following os.Args:\n\t// [\"brig\", \"stage\", \"--generate-bash-completion\"]\n\t//\n\t// Because of that we do no prefix completion right now.\n\t// We can probably tweak autcomplete/{z,ba}sh_autcomplete to\n\t// somehow do this, but after 30mins of googling I give up for now.\n\t//\n\t// If you read this, I challenge you to do it better.\n\tdir, err := os.Getwd()\n\tif err != nil {\n\t\t// silent error.\n\t\treturn\n\t}\n\n\tchildren, err := ioutil.ReadDir(dir)\n\tif err != nil {\n\t\t// silent error.\n\t\treturn\n\t}\n\n\tfor _, child := range children {\n\t\tfmt.Println(child.Name())\n\t}\n}\n\nfunc completeSubcommands(ctx *cli.Context) {\n\tif command := findCurrentCommand(ctx); command != nil {\n\t\tfor _, subCmd := range command.Subcommands {\n\t\t\tfmt.Println(subCmd.Name)\n\t\t}\n\t}\n}\n\nfunc commandNotFound(ctx *cli.Context, cmdName string) {\n\t// Try to find the commands we need to look at for a suggestion.\n\t// We only want to show the user the relevant subcommands.\n\tcmdPath, lastGoodCmds := findLastGoodCommands(ctx)\n\n\t// Figure out if it was a toplevel command or if some subcommand\n\t// (like e.g. 'remote') was correct.\n\tbadCmd := color.RedString(cmdName)\n\tif cmdPath == nil {\n\t\t// A toplevel command was wrong:\n\t\tfmt.Printf(\"»%s« is not a valid command. \", badCmd)\n\t} else {\n\t\t// A command of a subcommand was wrong:\n\t\tlastGoodSubCmd := color.YellowString(strings.Join(cmdPath, \" \"))\n\t\tfmt.Printf(\"»%s« is not a valid subcommand of »%s«. \", badCmd, lastGoodSubCmd)\n\t}\n\n\t// Get a list of similar commands:\n\tsimilars := findSimilarCommands(cmdName, lastGoodCmds)\n\n\tswitch len(similars) {\n\tcase 0:\n\t\tfmt.Printf(\"\\n\")\n\tcase 1:\n\t\tsuggestion := color.GreenString(similars[0].name)\n\t\tfmt.Printf(\"Did you maybe mean »%s«?\\n\", suggestion)\n\tdefault:\n\t\tfmt.Println(\"\\n\\nDid you maybe mean one of those?\")\n\t\tfor _, similar := range similars {\n\t\t\tfmt.Printf(\" * %s\\n\", color.GreenString(similar.name))\n\t\t}\n\t}\n}\n" }, { "path": "cmd/tabwriter/example_test.go", "content": "// Copyright 2012 The Go Authors. All rights reserved.\n// Use of this source code is governed by a BSD-style\n// license that can be found in the LICENSE file.\n\npackage tabwriter_test\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"text/tabwriter\"\n)\n\nfunc ExampleWriter_Init() {\n\tw := new(tabwriter.Writer)\n\n\t// Format in tab-separated columns with a tab stop of 8.\n\tw.Init(os.Stdout, 0, 8, 0, '\\t', 0)\n\tfmt.Fprintln(w, \"a\\tb\\tc\\td\\t.\")\n\tfmt.Fprintln(w, \"123\\t12345\\t1234567\\t123456789\\t.\")\n\tfmt.Fprintln(w)\n\tw.Flush()\n\n\t// Format right-aligned in space-separated columns of minimal width 5\n\t// and at least one blank of padding (so wider column entries do not\n\t// touch each other).\n\tw.Init(os.Stdout, 5, 0, 1, ' ', tabwriter.AlignRight)\n\tfmt.Fprintln(w, \"a\\tb\\tc\\td\\t.\")\n\tfmt.Fprintln(w, \"123\\t12345\\t1234567\\t123456789\\t.\")\n\tfmt.Fprintln(w)\n\tw.Flush()\n\n\t// output:\n\t// a\tb\tc\td\t\t.\n\t// 123\t12345\t1234567\t123456789\t.\n\t//\n\t// a b c d.\n\t// 123 12345 1234567 123456789.\n}\n\nfunc Example_elastic() {\n\t// Observe how the b's and the d's, despite appearing in the\n\t// second cell of each line, belong to different columns.\n\tw := tabwriter.NewWriter(os.Stdout, 0, 0, 1, '.', tabwriter.AlignRight|tabwriter.Debug)\n\tfmt.Fprintln(w, \"a\\tb\\tc\")\n\tfmt.Fprintln(w, \"aa\\tbb\\tcc\")\n\tfmt.Fprintln(w, \"aaa\\t\") // trailing tab\n\tfmt.Fprintln(w, \"aaaa\\tdddd\\teeee\")\n\tw.Flush()\n\n\t// output:\n\t// ....a|..b|c\n\t// ...aa|.bb|cc\n\t// ..aaa|\n\t// .aaaa|.dddd|eeee\n}\n\nfunc Example_trailingTab() {\n\t// Observe that the third line has no trailing tab,\n\t// so its final cell is not part of an aligned column.\n\tconst padding = 3\n\tw := tabwriter.NewWriter(os.Stdout, 0, 0, padding, '-', tabwriter.AlignRight|tabwriter.Debug)\n\tfmt.Fprintln(w, \"a\\tb\\taligned\\t\")\n\tfmt.Fprintln(w, \"aa\\tbb\\taligned\\t\")\n\tfmt.Fprintln(w, \"aaa\\tbbb\\tunaligned\") // no trailing tab\n\tfmt.Fprintln(w, \"aaaa\\tbbbb\\taligned\\t\")\n\tw.Flush()\n\n\t// output:\n\t// ------a|------b|---aligned|\n\t// -----aa|-----bb|---aligned|\n\t// ----aaa|----bbb|unaligned\n\t// ---aaaa|---bbbb|---aligned|\n}\n" }, { "path": "cmd/tabwriter/tabwriter.go", "content": "// Copyright 2009 The Go Authors. All rights reserved.\n// Use of this source code is governed by a BSD-style\n// license that can be found in the LICENSE file.\n\n// Package tabwriter implements a write filter (tabwriter.Writer) that\n// translates tabbed columns in input into properly aligned text.\n//\n// The package is using the Elastic Tabstops algorithm described at\n// http://nickgravgaard.com/elastictabstops/index.html.\n//\n// The text/tabwriter package is frozen and is not accepting new features.\npackage tabwriter\n\nimport (\n\t\"bytes\"\n\t\"io\"\n\t\"unicode/utf8\"\n)\n\n// ----------------------------------------------------------------------------\n// Filter implementation\n\n// A cell represents a segment of text terminated by tabs or line breaks.\n// The text itself is stored in a separate buffer; cell only describes the\n// segment's size in bytes, its width in runes, and whether it's an htab\n// ('\\t') terminated cell.\n//\ntype cell struct {\n\tsize int // cell size in bytes\n\twidth int // cell width in runes\n\thtab bool // true if the cell is terminated by an htab ('\\t')\n}\n\n// A Writer is a filter that inserts padding around tab-delimited\n// columns in its input to align them in the output.\n//\n// The Writer treats incoming bytes as UTF-8-encoded text consisting\n// of cells terminated by horizontal ('\\t') or vertical ('\\v') tabs,\n// and newline ('\\n') or formfeed ('\\f') characters; both newline and\n// formfeed act as line breaks.\n//\n// Tab-terminated cells in contiguous lines constitute a column. The\n// Writer inserts padding as needed to make all cells in a column have\n// the same width, effectively aligning the columns. It assumes that\n// all characters have the same width, except for tabs for which a\n// tabwidth must be specified. Column cells must be tab-terminated, not\n// tab-separated: non-tab terminated trailing text at the end of a line\n// forms a cell but that cell is not part of an aligned column.\n// For instance, in this example (where | stands for a horizontal tab):\n//\n//\taaaa|bbb|d\n//\taa |b |dd\n//\ta |\n//\taa |cccc|eee\n//\n// the b and c are in distinct columns (the b column is not contiguous\n// all the way). The d and e are not in a column at all (there's no\n// terminating tab, nor would the column be contiguous).\n//\n// The Writer assumes that all Unicode code points have the same width;\n// this may not be true in some fonts or if the string contains combining\n// characters.\n//\n// If DiscardEmptyColumns is set, empty columns that are terminated\n// entirely by vertical (or \"soft\") tabs are discarded. Columns\n// terminated by horizontal (or \"hard\") tabs are not affected by\n// this flag.\n//\n// If a Writer is configured to filter HTML, HTML tags and entities\n// are passed through. The widths of tags and entities are\n// assumed to be zero (tags) and one (entities) for formatting purposes.\n//\n// A segment of text may be escaped by bracketing it with Escape\n// characters. The tabwriter passes escaped text segments through\n// unchanged. In particular, it does not interpret any tabs or line\n// breaks within the segment. If the StripEscape flag is set, the\n// Escape characters are stripped from the output; otherwise they\n// are passed through as well. For the purpose of formatting, the\n// width of the escaped text is always computed excluding the Escape\n// characters.\n//\n// The formfeed character acts like a newline but it also terminates\n// all columns in the current line (effectively calling Flush). Tab-\n// terminated cells in the next line start new columns. Unless found\n// inside an HTML tag or inside an escaped text segment, formfeed\n// characters appear as newlines in the output.\n//\n// The Writer must buffer input internally, because proper spacing\n// of one line may depend on the cells in future lines. Clients must\n// call Flush when done calling Write.\n//\ntype Writer struct {\n\t// configuration\n\toutput io.Writer\n\tminwidth int\n\ttabwidth int\n\tpadding int\n\tpadbytes [8]byte\n\tflags uint\n\n\t// current state\n\tbuf bytes.Buffer // collected text excluding tabs or line breaks\n\tpos int // buffer position up to which cell.width of incomplete cell has been computed\n\tcell cell // current incomplete cell; cell.width is up to buf[pos] excluding ignored sections\n\tendChar byte // terminating char of escaped sequence (Escape for escapes, '>', ';' for HTML tags/entities, or 0)\n\tlines [][]cell // list of lines; each line is a list of cells\n\twidths []int // list of column widths in runes - re-used during formatting\n}\n\nfunc (b *Writer) addLine() { b.lines = append(b.lines, []cell{}) }\n\n// Reset the current state.\nfunc (b *Writer) reset() {\n\tb.buf.Reset()\n\tb.pos = 0\n\tb.cell = cell{}\n\tb.endChar = 0\n\tb.lines = b.lines[0:0]\n\tb.widths = b.widths[0:0]\n\tb.addLine()\n}\n\n// Internal representation (current state):\n//\n// - all text written is appended to buf; tabs and line breaks are stripped away\n// - at any given time there is a (possibly empty) incomplete cell at the end\n// (the cell starts after a tab or line break)\n// - cell.size is the number of bytes belonging to the cell so far\n// - cell.width is text width in runes of that cell from the start of the cell to\n// position pos; html tags and entities are excluded from this width if html\n// filtering is enabled\n// - the sizes and widths of processed text are kept in the lines list\n// which contains a list of cells for each line\n// - the widths list is a temporary list with current widths used during\n// formatting; it is kept in Writer because it's re-used\n//\n// |<---------- size ---------->|\n// | |\n// |<- width ->|<- ignored ->| |\n// | | | |\n// [---processed---tab------------......]\n// ^ ^ ^\n// | | |\n// buf start of incomplete cell pos\n\n// Formatting can be controlled with these flags.\nconst (\n\t// Ignore html tags and treat entities (starting with '&'\n\t// and ending in ';') as single characters (width = 1).\n\tFilterHTML uint = 1 << iota\n\n\t// Strip Escape characters bracketing escaped text segments\n\t// instead of passing them through unchanged with the text.\n\tStripEscape\n\n\t// Force right-alignment of cell content.\n\t// Default is left-alignment.\n\tAlignRight\n\n\t// Handle empty columns as if they were not present in\n\t// the input in the first place.\n\tDiscardEmptyColumns\n\n\t// Always use tabs for indentation columns (i.e., padding of\n\t// leading empty cells on the left) independent of padchar.\n\tTabIndent\n\n\t// Print a vertical bar ('|') between columns (after formatting).\n\t// Discarded columns appear as zero-width columns (\"||\").\n\tDebug\n)\n\n// Init initiliazes a Writer. The first parameter (output)\n// specifies the filter output. The remaining parameters control the formatting:\n//\n//\tminwidth\tminimal cell width including any padding\n//\ttabwidth\twidth of tab characters (equivalent number of spaces)\n//\tpadding\t\tpadding added to a cell before computing its width\n//\tpadchar\t\tASCII char used for padding\n//\t\t\tif padchar == '\\t', the Writer will assume that the\n//\t\t\twidth of a '\\t' in the formatted output is tabwidth,\n//\t\t\tand cells are left-aligned independent of align_left\n//\t\t\t(for correct-looking results, tabwidth must correspond\n//\t\t\tto the tab width in the viewer displaying the result)\n//\tflags\t\tformatting control\n//\nfunc (b *Writer) Init(output io.Writer, minwidth, tabwidth, padding int, padchar byte, flags uint) *Writer {\n\tif minwidth < 0 || tabwidth < 0 || padding < 0 {\n\t\tpanic(\"negative minwidth, tabwidth, or padding\")\n\t}\n\tb.output = output\n\tb.minwidth = minwidth\n\tb.tabwidth = tabwidth\n\tb.padding = padding\n\tfor i := range b.padbytes {\n\t\tb.padbytes[i] = padchar\n\t}\n\tif padchar == '\\t' {\n\t\t// tab padding enforces left-alignment\n\t\tflags &^= AlignRight\n\t}\n\tb.flags = flags\n\n\tb.reset()\n\n\treturn b\n}\n\n// debugging support (keep code around)\nfunc (b *Writer) dump() {\n\tpos := 0\n\tfor i, line := range b.lines {\n\t\tprint(\"(\", i, \") \")\n\t\tfor _, c := range line {\n\t\t\tprint(\"[\", string(b.buf.Bytes()[pos:pos+c.size]), \"]\")\n\t\t\tpos += c.size\n\t\t}\n\t\tprint(\"\\n\")\n\t}\n\tprint(\"\\n\")\n}\n\n// local error wrapper so we can distinguish errors we want to return\n// as errors from genuine panics (which we don't want to return as errors)\ntype osError struct {\n\terr error\n}\n\nfunc (b *Writer) write0(buf []byte) {\n\tn, err := b.output.Write(buf)\n\tif n != len(buf) && err == nil {\n\t\terr = io.ErrShortWrite\n\t}\n\tif err != nil {\n\t\tpanic(osError{err})\n\t}\n}\n\nfunc (b *Writer) writeN(src []byte, n int) {\n\tfor n > len(src) {\n\t\tb.write0(src)\n\t\tn -= len(src)\n\t}\n\tb.write0(src[0:n])\n}\n\nvar (\n\tnewline = []byte{'\\n'}\n\ttabs = []byte(\"\\t\\t\\t\\t\\t\\t\\t\\t\")\n)\n\nfunc (b *Writer) writePadding(textw, cellw int, useTabs bool) {\n\tif b.padbytes[0] == '\\t' || useTabs {\n\t\t// padding is done with tabs\n\t\tif b.tabwidth == 0 {\n\t\t\treturn // tabs have no width - can't do any padding\n\t\t}\n\t\t// make cellw the smallest multiple of b.tabwidth\n\t\tcellw = (cellw + b.tabwidth - 1) / b.tabwidth * b.tabwidth\n\t\tn := cellw - textw // amount of padding\n\t\tif n < 0 {\n\t\t\tpanic(\"internal error\")\n\t\t}\n\t\tb.writeN(tabs, (n+b.tabwidth-1)/b.tabwidth)\n\t\treturn\n\t}\n\n\t// padding is done with non-tab characters\n\tb.writeN(b.padbytes[0:], cellw-textw)\n}\n\nvar vbar = []byte{'|'}\n\nfunc (b *Writer) writeLines(pos0 int, line0, line1 int) (pos int) {\n\tpos = pos0\n\tfor i := line0; i < line1; i++ {\n\t\tline := b.lines[i]\n\n\t\t// if TabIndent is set, use tabs to pad leading empty cells\n\t\tuseTabs := b.flags&TabIndent != 0\n\n\t\tfor j, c := range line {\n\t\t\tif j > 0 && b.flags&Debug != 0 {\n\t\t\t\t// indicate column break\n\t\t\t\tb.write0(vbar)\n\t\t\t}\n\n\t\t\tif c.size == 0 {\n\t\t\t\t// empty cell\n\t\t\t\tif j < len(b.widths) {\n\t\t\t\t\tb.writePadding(c.width, b.widths[j], useTabs)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\t// non-empty cell\n\t\t\t\tuseTabs = false\n\t\t\t\tif b.flags&AlignRight == 0 { // align left\n\t\t\t\t\tb.write0(b.buf.Bytes()[pos : pos+c.size])\n\t\t\t\t\tpos += c.size\n\t\t\t\t\tif j < len(b.widths) {\n\t\t\t\t\t\tb.writePadding(c.width, b.widths[j], false)\n\t\t\t\t\t}\n\t\t\t\t} else { // align right\n\t\t\t\t\tif j < len(b.widths) {\n\t\t\t\t\t\tb.writePadding(c.width, b.widths[j], false)\n\t\t\t\t\t}\n\t\t\t\t\tb.write0(b.buf.Bytes()[pos : pos+c.size])\n\t\t\t\t\tpos += c.size\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\tif i+1 == len(b.lines) {\n\t\t\t// last buffered line - we don't have a newline, so just write\n\t\t\t// any outstanding buffered data\n\t\t\tb.write0(b.buf.Bytes()[pos : pos+b.cell.size])\n\t\t\tpos += b.cell.size\n\t\t} else {\n\t\t\t// not the last line - write newline\n\t\t\tb.write0(newline)\n\t\t}\n\t}\n\treturn\n}\n\n// Format the text between line0 and line1 (excluding line1); pos\n// is the buffer position corresponding to the beginning of line0.\n// Returns the buffer position corresponding to the beginning of\n// line1 and an error, if any.\n//\nfunc (b *Writer) format(pos0 int, line0, line1 int) (pos int) {\n\tpos = pos0\n\tcolumn := len(b.widths)\n\tfor this := line0; this < line1; this++ {\n\t\tline := b.lines[this]\n\n\t\tif column < len(line)-1 {\n\t\t\t// cell exists in this column => this line\n\t\t\t// has more cells than the previous line\n\t\t\t// (the last cell per line is ignored because cells are\n\t\t\t// tab-terminated; the last cell per line describes the\n\t\t\t// text before the newline/formfeed and does not belong\n\t\t\t// to a column)\n\n\t\t\t// print unprinted lines until beginning of block\n\t\t\tpos = b.writeLines(pos, line0, this)\n\t\t\tline0 = this\n\n\t\t\t// column block begin\n\t\t\twidth := b.minwidth // minimal column width\n\t\t\tdiscardable := true // true if all cells in this column are empty and \"soft\"\n\t\t\tfor ; this < line1; this++ {\n\t\t\t\tline = b.lines[this]\n\t\t\t\tif column < len(line)-1 {\n\t\t\t\t\t// cell exists in this column\n\t\t\t\t\tc := line[column]\n\t\t\t\t\t// update width\n\t\t\t\t\tif w := c.width + b.padding; w > width {\n\t\t\t\t\t\twidth = w\n\t\t\t\t\t}\n\t\t\t\t\t// update discardable\n\t\t\t\t\tif c.width > 0 || c.htab {\n\t\t\t\t\t\tdiscardable = false\n\t\t\t\t\t}\n\t\t\t\t} else {\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t}\n\t\t\t// column block end\n\n\t\t\t// discard empty columns if necessary\n\t\t\tif discardable && b.flags&DiscardEmptyColumns != 0 {\n\t\t\t\twidth = 0\n\t\t\t}\n\n\t\t\t// format and print all columns to the right of this column\n\t\t\t// (we know the widths of this column and all columns to the left)\n\t\t\tb.widths = append(b.widths, width) // push width\n\t\t\tpos = b.format(pos, line0, this)\n\t\t\tb.widths = b.widths[0 : len(b.widths)-1] // pop width\n\t\t\tline0 = this\n\t\t}\n\t}\n\n\t// print unprinted lines until end\n\treturn b.writeLines(pos, line0, line1)\n}\n\n// Append text to current cell.\nfunc (b *Writer) append(text []byte) {\n\tb.buf.Write(text)\n\tb.cell.size += len(text)\n}\n\n// Update the cell width.\nfunc (b *Writer) updateWidth() {\n\tb.cell.width += utf8.RuneCount(b.buf.Bytes()[b.pos:b.buf.Len()])\n\tb.pos = b.buf.Len()\n}\n\n// Escape escapes a text segment.\n// For instance, the tab in this string \"Ignore this tab: \\xff\\t\\xff\"\n// does not terminate a cell and constitutes a single character of\n// width one for formatting purposes.\n//\n// The value 0xff was chosen because it cannot appear in a valid UTF-8 sequence.\n//\nconst Escape = '\\xff'\n\n// ColorStart defines a byte sequence that indicates the start of terminal colors.\nconst ColorStart = '\\x1B'\n\n// Start escaped mode.\nfunc (b *Writer) startEscape(ch byte) {\n\tswitch ch {\n\tcase Escape:\n\t\tb.endChar = Escape\n\tcase '<':\n\t\tb.endChar = '>'\n\tcase '&':\n\t\tb.endChar = ';'\n\tcase ColorStart:\n\t\tb.endChar = 'm'\n\t}\n}\n\n// Terminate escaped mode. If the escaped text was an HTML tag, its width\n// is assumed to be zero for formatting purposes; if it was an HTML entity,\n// its width is assumed to be one. In all other cases, the width is the\n// unicode width of the text.\n//\nfunc (b *Writer) endEscape() {\n\tswitch b.endChar {\n\tcase Escape:\n\t\tb.updateWidth()\n\t\tif b.flags&StripEscape == 0 {\n\t\t\tb.cell.width -= 2 // don't count the Escape chars\n\t\t}\n\tcase 'm': // color escape do not count\n\tcase '>': // tag of zero width\n\tcase ';':\n\t\tb.cell.width++ // entity, count as one rune\n\t}\n\tb.pos = b.buf.Len()\n\tb.endChar = 0\n}\n\n// Terminate the current cell by adding it to the list of cells of the\n// current line. Returns the number of cells in that line.\n//\nfunc (b *Writer) terminateCell(htab bool) int {\n\tb.cell.htab = htab\n\tline := &b.lines[len(b.lines)-1]\n\t*line = append(*line, b.cell)\n\tb.cell = cell{}\n\treturn len(*line)\n}\n\nfunc handlePanic(err *error, op string) {\n\tif e := recover(); e != nil {\n\t\tif nerr, ok := e.(osError); ok {\n\t\t\t*err = nerr.err\n\t\t\treturn\n\t\t}\n\t\tpanic(\"tabwriter: panic during \" + op)\n\t}\n}\n\n// Flush should be called after the last call to Write to ensure\n// that any data buffered in the Writer is written to output. Any\n// incomplete escape sequence at the end is considered\n// complete for formatting purposes.\nfunc (b *Writer) Flush() error {\n\treturn b.flush()\n}\n\nfunc (b *Writer) flush() (err error) {\n\tdefer b.reset() // even in the presence of errors\n\tdefer handlePanic(&err, \"Flush\")\n\n\t// add current cell if not empty\n\tif b.cell.size > 0 {\n\t\tif b.endChar != 0 {\n\t\t\t// inside escape - terminate it even if incomplete\n\t\t\tb.endEscape()\n\t\t}\n\t\tb.terminateCell(false)\n\t}\n\n\t// format contents of buffer\n\tb.format(0, 0, len(b.lines))\n\treturn nil\n}\n\nvar hbar = []byte(\"---\\n\")\n\n// Write writes buf to the writer b.\n// The only errors returned are ones encountered\n// while writing to the underlying output stream.\n//\nfunc (b *Writer) Write(buf []byte) (n int, err error) {\n\tdefer handlePanic(&err, \"Write\")\n\n\t// split text into cells\n\tn = 0\n\tfor i, ch := range buf {\n\t\tif b.endChar == 0 {\n\t\t\t// outside escape\n\t\t\tswitch ch {\n\t\t\tcase '\\t', '\\v', '\\n', '\\f':\n\t\t\t\t// end of cell\n\t\t\t\tb.append(buf[n:i])\n\t\t\t\tb.updateWidth()\n\t\t\t\tn = i + 1 // ch consumed\n\t\t\t\tncells := b.terminateCell(ch == '\\t')\n\t\t\t\tif ch == '\\n' || ch == '\\f' {\n\t\t\t\t\t// terminate line\n\t\t\t\t\tb.addLine()\n\t\t\t\t\tif ch == '\\f' || ncells == 1 {\n\t\t\t\t\t\t// A '\\f' always forces a flush. Otherwise, if the previous\n\t\t\t\t\t\t// line has only one cell which does not have an impact on\n\t\t\t\t\t\t// the formatting of the following lines (the last cell per\n\t\t\t\t\t\t// line is ignored by format()), thus we can flush the\n\t\t\t\t\t\t// Writer contents.\n\t\t\t\t\t\tif err = b.Flush(); err != nil {\n\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t}\n\t\t\t\t\t\tif ch == '\\f' && b.flags&Debug != 0 {\n\t\t\t\t\t\t\t// indicate section break\n\t\t\t\t\t\t\tb.write0(hbar)\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\tcase Escape:\n\t\t\t\t// start of escaped sequence\n\t\t\t\tb.append(buf[n:i])\n\t\t\t\tb.updateWidth()\n\t\t\t\tn = i\n\t\t\t\tif b.flags&StripEscape != 0 {\n\t\t\t\t\tn++ // strip Escape\n\t\t\t\t}\n\t\t\t\tb.startEscape(Escape)\n\n\t\t\tcase ColorStart:\n\t\t\t\tb.append(buf[n:i])\n\t\t\t\tb.updateWidth()\n\t\t\t\tn = i\n\t\t\t\tb.startEscape(ch)\n\t\t\tcase '<', '&':\n\t\t\t\t// possibly an html tag/entity\n\t\t\t\tif b.flags&FilterHTML != 0 {\n\t\t\t\t\t// begin of tag/entity\n\t\t\t\t\tb.append(buf[n:i])\n\t\t\t\t\tb.updateWidth()\n\t\t\t\t\tn = i\n\t\t\t\t\tb.startEscape(ch)\n\t\t\t\t}\n\t\t\t}\n\n\t\t} else {\n\t\t\t// inside escape\n\t\t\tif ch == b.endChar {\n\t\t\t\t// end of tag/entity\n\t\t\t\tj := i + 1\n\t\t\t\tif ch == Escape && b.flags&StripEscape != 0 {\n\t\t\t\t\tj = i // strip Escape\n\t\t\t\t}\n\t\t\t\tb.append(buf[n:j])\n\t\t\t\tn = i + 1 // ch consumed\n\t\t\t\tb.endEscape()\n\t\t\t}\n\t\t}\n\t}\n\n\t// append leftover text\n\tb.append(buf[n:])\n\tn = len(buf)\n\treturn\n}\n\n// NewWriter allocates and initializes a new tabwriter.Writer.\n// The parameters are the same as for the Init function.\n//\nfunc NewWriter(output io.Writer, minwidth, tabwidth, padding int, padchar byte, flags uint) *Writer {\n\treturn new(Writer).Init(output, minwidth, tabwidth, padding, padchar, flags)\n}\n" }, { "path": "cmd/tabwriter/tabwriter_test.go", "content": "// Copyright 2009 The Go Authors. All rights reserved.\n// Use of this source code is governed by a BSD-style\n// license that can be found in the LICENSE file.\n\npackage tabwriter_test\n\nimport (\n\t\"io\"\n\t\"testing\"\n\t. \"text/tabwriter\"\n)\n\ntype buffer struct {\n\ta []byte\n}\n\nfunc (b *buffer) init(n int) { b.a = make([]byte, 0, n) }\n\nfunc (b *buffer) clear() { b.a = b.a[0:0] }\n\nfunc (b *buffer) Write(buf []byte) (written int, err error) {\n\tn := len(b.a)\n\tm := len(buf)\n\tif n+m <= cap(b.a) {\n\t\tb.a = b.a[0 : n+m]\n\t\tfor i := 0; i < m; i++ {\n\t\t\tb.a[n+i] = buf[i]\n\t\t}\n\t} else {\n\t\tpanic(\"buffer.Write: buffer too small\")\n\t}\n\treturn len(buf), nil\n}\n\nfunc (b *buffer) String() string { return string(b.a) }\n\nfunc write(t *testing.T, testname string, w *Writer, src string) {\n\twritten, err := io.WriteString(w, src)\n\tif err != nil {\n\t\tt.Errorf(\"--- test: %s\\n--- src:\\n%q\\n--- write error: %v\\n\", testname, src, err)\n\t}\n\tif written != len(src) {\n\t\tt.Errorf(\"--- test: %s\\n--- src:\\n%q\\n--- written = %d, len(src) = %d\\n\", testname, src, written, len(src))\n\t}\n}\n\nfunc verify(t *testing.T, testname string, w *Writer, b *buffer, src, expected string) {\n\terr := w.Flush()\n\tif err != nil {\n\t\tt.Errorf(\"--- test: %s\\n--- src:\\n%q\\n--- flush error: %v\\n\", testname, src, err)\n\t}\n\n\tres := b.String()\n\tif res != expected {\n\t\tt.Errorf(\"--- test: %s\\n--- src:\\n%q\\n--- found:\\n%q\\n--- expected:\\n%q\\n\", testname, src, res, expected)\n\t}\n}\n\nfunc check(t *testing.T, testname string, minwidth, tabwidth, padding int, padchar byte, flags uint, src, expected string) {\n\tvar b buffer\n\tb.init(1000)\n\n\tvar w Writer\n\tw.Init(&b, minwidth, tabwidth, padding, padchar, flags)\n\n\t// write all at once\n\ttitle := testname + \" (written all at once)\"\n\tb.clear()\n\twrite(t, title, &w, src)\n\tverify(t, title, &w, &b, src, expected)\n\n\t// write byte-by-byte\n\ttitle = testname + \" (written byte-by-byte)\"\n\tb.clear()\n\tfor i := 0; i < len(src); i++ {\n\t\twrite(t, title, &w, src[i:i+1])\n\t}\n\tverify(t, title, &w, &b, src, expected)\n\n\t// write using Fibonacci slice sizes\n\ttitle = testname + \" (written in fibonacci slices)\"\n\tb.clear()\n\tfor i, d := 0, 0; i < len(src); {\n\t\twrite(t, title, &w, src[i:i+d])\n\t\ti, d = i+d, d+1\n\t\tif i+d > len(src) {\n\t\t\td = len(src) - i\n\t\t}\n\t}\n\tverify(t, title, &w, &b, src, expected)\n}\n\nvar tests = []struct {\n\ttestname string\n\tminwidth, tabwidth, padding int\n\tpadchar byte\n\tflags uint\n\tsrc, expected string\n}{\n\t{\n\t\t\"1a\",\n\t\t8, 0, 1, '.', 0,\n\t\t\"\",\n\t\t\"\",\n\t},\n\n\t{\n\t\t\"1a debug\",\n\t\t8, 0, 1, '.', Debug,\n\t\t\"\",\n\t\t\"\",\n\t},\n\n\t{\n\t\t\"1b esc stripped\",\n\t\t8, 0, 1, '.', StripEscape,\n\t\t\"\\xff\\xff\",\n\t\t\"\",\n\t},\n\n\t{\n\t\t\"1b esc\",\n\t\t8, 0, 1, '.', 0,\n\t\t\"\\xff\\xff\",\n\t\t\"\\xff\\xff\",\n\t},\n\n\t{\n\t\t\"1c esc stripped\",\n\t\t8, 0, 1, '.', StripEscape,\n\t\t\"\\xff\\t\\xff\",\n\t\t\"\\t\",\n\t},\n\n\t{\n\t\t\"1c esc\",\n\t\t8, 0, 1, '.', 0,\n\t\t\"\\xff\\t\\xff\",\n\t\t\"\\xff\\t\\xff\",\n\t},\n\n\t{\n\t\t\"1d esc stripped\",\n\t\t8, 0, 1, '.', StripEscape,\n\t\t\"\\xff\\\"foo\\t\\n\\tbar\\\"\\xff\",\n\t\t\"\\\"foo\\t\\n\\tbar\\\"\",\n\t},\n\n\t{\n\t\t\"1d esc\",\n\t\t8, 0, 1, '.', 0,\n\t\t\"\\xff\\\"foo\\t\\n\\tbar\\\"\\xff\",\n\t\t\"\\xff\\\"foo\\t\\n\\tbar\\\"\\xff\",\n\t},\n\n\t{\n\t\t\"1e esc stripped\",\n\t\t8, 0, 1, '.', StripEscape,\n\t\t\"abc\\xff\\tdef\", // unterminated escape\n\t\t\"abc\\tdef\",\n\t},\n\n\t{\n\t\t\"1e esc\",\n\t\t8, 0, 1, '.', 0,\n\t\t\"abc\\xff\\tdef\", // unterminated escape\n\t\t\"abc\\xff\\tdef\",\n\t},\n\n\t{\n\t\t\"2\",\n\t\t8, 0, 1, '.', 0,\n\t\t\"\\n\\n\\n\",\n\t\t\"\\n\\n\\n\",\n\t},\n\n\t{\n\t\t\"3\",\n\t\t8, 0, 1, '.', 0,\n\t\t\"a\\nb\\nc\",\n\t\t\"a\\nb\\nc\",\n\t},\n\n\t{\n\t\t\"4a\",\n\t\t8, 0, 1, '.', 0,\n\t\t\"\\t\", // '\\t' terminates an empty cell on last line - nothing to print\n\t\t\"\",\n\t},\n\n\t{\n\t\t\"4b\",\n\t\t8, 0, 1, '.', AlignRight,\n\t\t\"\\t\", // '\\t' terminates an empty cell on last line - nothing to print\n\t\t\"\",\n\t},\n\n\t{\n\t\t\"5\",\n\t\t8, 0, 1, '.', 0,\n\t\t\"*\\t*\",\n\t\t\"*.......*\",\n\t},\n\n\t{\n\t\t\"5b\",\n\t\t8, 0, 1, '.', 0,\n\t\t\"*\\t*\\n\",\n\t\t\"*.......*\\n\",\n\t},\n\n\t{\n\t\t\"5c\",\n\t\t8, 0, 1, '.', 0,\n\t\t\"*\\t*\\t\",\n\t\t\"*.......*\",\n\t},\n\n\t{\n\t\t\"5c debug\",\n\t\t8, 0, 1, '.', Debug,\n\t\t\"*\\t*\\t\",\n\t\t\"*.......|*\",\n\t},\n\n\t{\n\t\t\"5d\",\n\t\t8, 0, 1, '.', AlignRight,\n\t\t\"*\\t*\\t\",\n\t\t\".......**\",\n\t},\n\n\t{\n\t\t\"6\",\n\t\t8, 0, 1, '.', 0,\n\t\t\"\\t\\n\",\n\t\t\"........\\n\",\n\t},\n\n\t{\n\t\t\"7a\",\n\t\t8, 0, 1, '.', 0,\n\t\t\"a) foo\",\n\t\t\"a) foo\",\n\t},\n\n\t{\n\t\t\"7b\",\n\t\t8, 0, 1, ' ', 0,\n\t\t\"b) foo\\tbar\",\n\t\t\"b) foo bar\",\n\t},\n\n\t{\n\t\t\"7c\",\n\t\t8, 0, 1, '.', 0,\n\t\t\"c) foo\\tbar\\t\",\n\t\t\"c) foo..bar\",\n\t},\n\n\t{\n\t\t\"7d\",\n\t\t8, 0, 1, '.', 0,\n\t\t\"d) foo\\tbar\\n\",\n\t\t\"d) foo..bar\\n\",\n\t},\n\n\t{\n\t\t\"7e\",\n\t\t8, 0, 1, '.', 0,\n\t\t\"e) foo\\tbar\\t\\n\",\n\t\t\"e) foo..bar.....\\n\",\n\t},\n\n\t{\n\t\t\"7f\",\n\t\t8, 0, 1, '.', FilterHTML,\n\t\t\"f) f<o\\tbar\\t\\n\",\n\t\t\"f) f<o..bar.....\\n\",\n\t},\n\n\t{\n\t\t\"7g\",\n\t\t8, 0, 1, '.', FilterHTML,\n\t\t\"g) f<o\\tbar\\t non-terminated entity &\",\n\t\t\"g) f<o..bar..... non-terminated entity &\",\n\t},\n\n\t{\n\t\t\"7g debug\",\n\t\t8, 0, 1, '.', FilterHTML | Debug,\n\t\t\"g) f<o\\tbar\\t non-terminated entity &\",\n\t\t\"g) f<o..|bar.....| non-terminated entity &\",\n\t},\n\n\t{\n\t\t\"8\",\n\t\t8, 0, 1, '*', 0,\n\t\t\"Hello, world!\\n\",\n\t\t\"Hello, world!\\n\",\n\t},\n\n\t{\n\t\t\"9a\",\n\t\t1, 0, 0, '.', 0,\n\t\t\"1\\t2\\t3\\t4\\n\" +\n\t\t\t\"11\\t222\\t3333\\t44444\\n\",\n\n\t\t\"1.2..3...4\\n\" +\n\t\t\t\"11222333344444\\n\",\n\t},\n\n\t{\n\t\t\"9b\",\n\t\t1, 0, 0, '.', FilterHTML,\n\t\t\"1\\t2\\t3\\t4\\n\" + // \\f inside HTML is ignored\n\t\t\t\"11\\t222\\t3333\\t44444\\n\",\n\n\t\t\"1.2..3...4\\n\" +\n\t\t\t\"11222333344444\\n\",\n\t},\n\n\t{\n\t\t\"9c\",\n\t\t1, 0, 0, '.', 0,\n\t\t\"1\\t2\\t3\\t4\\f\" + // \\f causes a newline and flush\n\t\t\t\"11\\t222\\t3333\\t44444\\n\",\n\n\t\t\"1234\\n\" +\n\t\t\t\"11222333344444\\n\",\n\t},\n\n\t{\n\t\t\"9c debug\",\n\t\t1, 0, 0, '.', Debug,\n\t\t\"1\\t2\\t3\\t4\\f\" + // \\f causes a newline and flush\n\t\t\t\"11\\t222\\t3333\\t44444\\n\",\n\n\t\t\"1|2|3|4\\n\" +\n\t\t\t\"---\\n\" +\n\t\t\t\"11|222|3333|44444\\n\",\n\t},\n\n\t{\n\t\t\"10a\",\n\t\t5, 0, 0, '.', 0,\n\t\t\"1\\t2\\t3\\t4\\n\",\n\t\t\"1....2....3....4\\n\",\n\t},\n\n\t{\n\t\t\"10b\",\n\t\t5, 0, 0, '.', 0,\n\t\t\"1\\t2\\t3\\t4\\t\\n\",\n\t\t\"1....2....3....4....\\n\",\n\t},\n\n\t{\n\t\t\"11\",\n\t\t8, 0, 1, '.', 0,\n\t\t\"本\\tb\\tc\\n\" +\n\t\t\t\"aa\\t\\u672c\\u672c\\u672c\\tcccc\\tddddd\\n\" +\n\t\t\t\"aaa\\tbbbb\\n\",\n\n\t\t\"本.......b.......c\\n\" +\n\t\t\t\"aa......本本本.....cccc....ddddd\\n\" +\n\t\t\t\"aaa.....bbbb\\n\",\n\t},\n\n\t{\n\t\t\"12a\",\n\t\t8, 0, 1, ' ', AlignRight,\n\t\t\"a\\tè\\tc\\t\\n\" +\n\t\t\t\"aa\\tèèè\\tcccc\\tddddd\\t\\n\" +\n\t\t\t\"aaa\\tèèèè\\t\\n\",\n\n\t\t\" a è c\\n\" +\n\t\t\t\" aa èèè cccc ddddd\\n\" +\n\t\t\t\" aaa èèèè\\n\",\n\t},\n\n\t{\n\t\t\"12b\",\n\t\t2, 0, 0, ' ', 0,\n\t\t\"a\\tb\\tc\\n\" +\n\t\t\t\"aa\\tbbb\\tcccc\\n\" +\n\t\t\t\"aaa\\tbbbb\\n\",\n\n\t\t\"a b c\\n\" +\n\t\t\t\"aa bbbcccc\\n\" +\n\t\t\t\"aaabbbb\\n\",\n\t},\n\n\t{\n\t\t\"12c\",\n\t\t8, 0, 1, '_', 0,\n\t\t\"a\\tb\\tc\\n\" +\n\t\t\t\"aa\\tbbb\\tcccc\\n\" +\n\t\t\t\"aaa\\tbbbb\\n\",\n\n\t\t\"a_______b_______c\\n\" +\n\t\t\t\"aa______bbb_____cccc\\n\" +\n\t\t\t\"aaa_____bbbb\\n\",\n\t},\n\n\t{\n\t\t\"13a\",\n\t\t4, 0, 1, '-', 0,\n\t\t\"4444\\t日本語\\t22\\t1\\t333\\n\" +\n\t\t\t\"999999999\\t22\\n\" +\n\t\t\t\"7\\t22\\n\" +\n\t\t\t\"\\t\\t\\t88888888\\n\" +\n\t\t\t\"\\n\" +\n\t\t\t\"666666\\t666666\\t666666\\t4444\\n\" +\n\t\t\t\"1\\t1\\t999999999\\t0000000000\\n\",\n\n\t\t\"4444------日本語-22--1---333\\n\" +\n\t\t\t\"999999999-22\\n\" +\n\t\t\t\"7---------22\\n\" +\n\t\t\t\"------------------88888888\\n\" +\n\t\t\t\"\\n\" +\n\t\t\t\"666666-666666-666666----4444\\n\" +\n\t\t\t\"1------1------999999999-0000000000\\n\",\n\t},\n\n\t{\n\t\t\"13b\",\n\t\t4, 0, 3, '.', 0,\n\t\t\"4444\\t333\\t22\\t1\\t333\\n\" +\n\t\t\t\"999999999\\t22\\n\" +\n\t\t\t\"7\\t22\\n\" +\n\t\t\t\"\\t\\t\\t88888888\\n\" +\n\t\t\t\"\\n\" +\n\t\t\t\"666666\\t666666\\t666666\\t4444\\n\" +\n\t\t\t\"1\\t1\\t999999999\\t0000000000\\n\",\n\n\t\t\"4444........333...22...1...333\\n\" +\n\t\t\t\"999999999...22\\n\" +\n\t\t\t\"7...........22\\n\" +\n\t\t\t\"....................88888888\\n\" +\n\t\t\t\"\\n\" +\n\t\t\t\"666666...666666...666666......4444\\n\" +\n\t\t\t\"1........1........999999999...0000000000\\n\",\n\t},\n\n\t{\n\t\t\"13c\",\n\t\t8, 8, 1, '\\t', FilterHTML,\n\t\t\"4444\\t333\\t22\\t1\\t333\\n\" +\n\t\t\t\"999999999\\t22\\n\" +\n\t\t\t\"7\\t22\\n\" +\n\t\t\t\"\\t\\t\\t88888888\\n\" +\n\t\t\t\"\\n\" +\n\t\t\t\"666666\\t666666\\t666666\\t4444\\n\" +\n\t\t\t\"1\\t1\\t999999999\\t0000000000\\n\",\n\n\t\t\"4444\\t\\t333\\t22\\t1\\t333\\n\" +\n\t\t\t\"999999999\\t22\\n\" +\n\t\t\t\"7\\t\\t22\\n\" +\n\t\t\t\"\\t\\t\\t\\t88888888\\n\" +\n\t\t\t\"\\n\" +\n\t\t\t\"666666\\t666666\\t666666\\t\\t4444\\n\" +\n\t\t\t\"1\\t1\\t999999999\\t0000000000\\n\",\n\t},\n\n\t{\n\t\t\"14\",\n\t\t1, 0, 2, ' ', AlignRight,\n\t\t\".0\\t.3\\t2.4\\t-5.1\\t\\n\" +\n\t\t\t\"23.0\\t12345678.9\\t2.4\\t-989.4\\t\\n\" +\n\t\t\t\"5.1\\t12.0\\t2.4\\t-7.0\\t\\n\" +\n\t\t\t\".0\\t0.0\\t332.0\\t8908.0\\t\\n\" +\n\t\t\t\".0\\t-.3\\t456.4\\t22.1\\t\\n\" +\n\t\t\t\".0\\t1.2\\t44.4\\t-13.3\\t\\t\",\n\n\t\t\" .0 .3 2.4 -5.1\\n\" +\n\t\t\t\" 23.0 12345678.9 2.4 -989.4\\n\" +\n\t\t\t\" 5.1 12.0 2.4 -7.0\\n\" +\n\t\t\t\" .0 0.0 332.0 8908.0\\n\" +\n\t\t\t\" .0 -.3 456.4 22.1\\n\" +\n\t\t\t\" .0 1.2 44.4 -13.3\",\n\t},\n\n\t{\n\t\t\"14 debug\",\n\t\t1, 0, 2, ' ', AlignRight | Debug,\n\t\t\".0\\t.3\\t2.4\\t-5.1\\t\\n\" +\n\t\t\t\"23.0\\t12345678.9\\t2.4\\t-989.4\\t\\n\" +\n\t\t\t\"5.1\\t12.0\\t2.4\\t-7.0\\t\\n\" +\n\t\t\t\".0\\t0.0\\t332.0\\t8908.0\\t\\n\" +\n\t\t\t\".0\\t-.3\\t456.4\\t22.1\\t\\n\" +\n\t\t\t\".0\\t1.2\\t44.4\\t-13.3\\t\\t\",\n\n\t\t\" .0| .3| 2.4| -5.1|\\n\" +\n\t\t\t\" 23.0| 12345678.9| 2.4| -989.4|\\n\" +\n\t\t\t\" 5.1| 12.0| 2.4| -7.0|\\n\" +\n\t\t\t\" .0| 0.0| 332.0| 8908.0|\\n\" +\n\t\t\t\" .0| -.3| 456.4| 22.1|\\n\" +\n\t\t\t\" .0| 1.2| 44.4| -13.3|\",\n\t},\n\n\t{\n\t\t\"15a\",\n\t\t4, 0, 0, '.', 0,\n\t\t\"a\\t\\tb\",\n\t\t\"a.......b\",\n\t},\n\n\t{\n\t\t\"15b\",\n\t\t4, 0, 0, '.', DiscardEmptyColumns,\n\t\t\"a\\t\\tb\", // htabs - do not discard column\n\t\t\"a.......b\",\n\t},\n\n\t{\n\t\t\"15c\",\n\t\t4, 0, 0, '.', DiscardEmptyColumns,\n\t\t\"a\\v\\vb\",\n\t\t\"a...b\",\n\t},\n\n\t{\n\t\t\"15d\",\n\t\t4, 0, 0, '.', AlignRight | DiscardEmptyColumns,\n\t\t\"a\\v\\vb\",\n\t\t\"...ab\",\n\t},\n\n\t{\n\t\t\"16a\",\n\t\t100, 100, 0, '\\t', 0,\n\t\t\"a\\tb\\t\\td\\n\" +\n\t\t\t\"a\\tb\\t\\td\\te\\n\" +\n\t\t\t\"a\\n\" +\n\t\t\t\"a\\tb\\tc\\td\\n\" +\n\t\t\t\"a\\tb\\tc\\td\\te\\n\",\n\n\t\t\"a\\tb\\t\\td\\n\" +\n\t\t\t\"a\\tb\\t\\td\\te\\n\" +\n\t\t\t\"a\\n\" +\n\t\t\t\"a\\tb\\tc\\td\\n\" +\n\t\t\t\"a\\tb\\tc\\td\\te\\n\",\n\t},\n\n\t{\n\t\t\"16b\",\n\t\t100, 100, 0, '\\t', DiscardEmptyColumns,\n\t\t\"a\\vb\\v\\vd\\n\" +\n\t\t\t\"a\\vb\\v\\vd\\ve\\n\" +\n\t\t\t\"a\\n\" +\n\t\t\t\"a\\vb\\vc\\vd\\n\" +\n\t\t\t\"a\\vb\\vc\\vd\\ve\\n\",\n\n\t\t\"a\\tb\\td\\n\" +\n\t\t\t\"a\\tb\\td\\te\\n\" +\n\t\t\t\"a\\n\" +\n\t\t\t\"a\\tb\\tc\\td\\n\" +\n\t\t\t\"a\\tb\\tc\\td\\te\\n\",\n\t},\n\n\t{\n\t\t\"16b debug\",\n\t\t100, 100, 0, '\\t', DiscardEmptyColumns | Debug,\n\t\t\"a\\vb\\v\\vd\\n\" +\n\t\t\t\"a\\vb\\v\\vd\\ve\\n\" +\n\t\t\t\"a\\n\" +\n\t\t\t\"a\\vb\\vc\\vd\\n\" +\n\t\t\t\"a\\vb\\vc\\vd\\ve\\n\",\n\n\t\t\"a\\t|b\\t||d\\n\" +\n\t\t\t\"a\\t|b\\t||d\\t|e\\n\" +\n\t\t\t\"a\\n\" +\n\t\t\t\"a\\t|b\\t|c\\t|d\\n\" +\n\t\t\t\"a\\t|b\\t|c\\t|d\\t|e\\n\",\n\t},\n\n\t{\n\t\t\"16c\",\n\t\t100, 100, 0, '\\t', DiscardEmptyColumns,\n\t\t\"a\\tb\\t\\td\\n\" + // hard tabs - do not discard column\n\t\t\t\"a\\tb\\t\\td\\te\\n\" +\n\t\t\t\"a\\n\" +\n\t\t\t\"a\\tb\\tc\\td\\n\" +\n\t\t\t\"a\\tb\\tc\\td\\te\\n\",\n\n\t\t\"a\\tb\\t\\td\\n\" +\n\t\t\t\"a\\tb\\t\\td\\te\\n\" +\n\t\t\t\"a\\n\" +\n\t\t\t\"a\\tb\\tc\\td\\n\" +\n\t\t\t\"a\\tb\\tc\\td\\te\\n\",\n\t},\n\n\t{\n\t\t\"16c debug\",\n\t\t100, 100, 0, '\\t', DiscardEmptyColumns | Debug,\n\t\t\"a\\tb\\t\\td\\n\" + // hard tabs - do not discard column\n\t\t\t\"a\\tb\\t\\td\\te\\n\" +\n\t\t\t\"a\\n\" +\n\t\t\t\"a\\tb\\tc\\td\\n\" +\n\t\t\t\"a\\tb\\tc\\td\\te\\n\",\n\n\t\t\"a\\t|b\\t|\\t|d\\n\" +\n\t\t\t\"a\\t|b\\t|\\t|d\\t|e\\n\" +\n\t\t\t\"a\\n\" +\n\t\t\t\"a\\t|b\\t|c\\t|d\\n\" +\n\t\t\t\"a\\t|b\\t|c\\t|d\\t|e\\n\",\n\t},\n}\n\nfunc Test(t *testing.T) {\n\tfor _, e := range tests {\n\t\tcheck(t, e.testname, e.minwidth, e.tabwidth, e.padding, e.padchar, e.flags, e.src, e.expected)\n\t}\n}\n\ntype panicWriter struct{}\n\nfunc (panicWriter) Write([]byte) (int, error) {\n\tpanic(\"cannot write\")\n}\n\nfunc wantPanicString(t *testing.T, want string) {\n\tif e := recover(); e != nil {\n\t\tgot, ok := e.(string)\n\t\tswitch {\n\t\tcase !ok:\n\t\t\tt.Errorf(\"got %v (%T), want panic string\", e, e)\n\t\tcase got != want:\n\t\t\tt.Errorf(\"wrong panic message: got %q, want %q\", got, want)\n\t\t}\n\t}\n}\n\nfunc TestPanicDuringFlush(t *testing.T) {\n\tdefer wantPanicString(t, \"tabwriter: panic during Flush\")\n\tvar p panicWriter\n\tw := new(Writer)\n\tw.Init(p, 0, 0, 5, ' ', 0)\n\tio.WriteString(w, \"a\")\n\tw.Flush()\n\tt.Errorf(\"failed to panic during Flush\")\n}\n\nfunc TestPanicDuringWrite(t *testing.T) {\n\tdefer wantPanicString(t, \"tabwriter: panic during Write\")\n\tvar p panicWriter\n\tw := new(Writer)\n\tw.Init(p, 0, 0, 5, ' ', 0)\n\tio.WriteString(w, \"a\\n\\n\") // the second \\n triggers a call to w.Write and thus a panic\n\tt.Errorf(\"failed to panic during Write\")\n}\n" }, { "path": "cmd/tree.go", "content": "package cmd\n\nimport (\n\t\"fmt\"\n\t\"sort\"\n\t\"strings\"\n\t\"unicode\"\n\n\t\"github.com/fatih/color\"\n\t\"github.com/sahib/brig/client\"\n)\n\nvar (\n\ttreeRunePipe = \"│\"\n\ttreeRuneTri = \"├\"\n\ttreeRuneBar = \"──\"\n\ttreeRuneCorner = \"└\"\n)\n\ntype treeNode struct {\n\tname string\n\torder []*treeNode\n\tchildren map[string]*treeNode\n\tisLast bool\n\tparent *treeNode\n\tdepth int\n\tentry client.StatInfo\n}\n\ntype treeCfg struct {\n\tshowPin bool\n\tformat func(n *treeNode) string\n}\n\n// This is a very stripped down version of util.Trie.Insert()\n// but with support for ordering the elements.\nfunc (n *treeNode) Insert(entry client.StatInfo) {\n\tparts := strings.Split(entry.Path, \"/\")\n\tif len(parts) > 0 && parts[0] == \"\" {\n\t\tparts = parts[1:]\n\t}\n\n\tcurr := n\n\tcurrParts := []string{\"\"}\n\n\tfor depth, name := range parts {\n\t\tif curr.children == nil {\n\t\t\tcurr.children = make(map[string]*treeNode)\n\t\t}\n\n\t\tcurrParts = append(currParts, name)\n\t\tcurrPath := strings.Join(currParts, \"/\")\n\n\t\tchild, ok := curr.children[name]\n\t\tif !ok {\n\t\t\tchildEntry := entry\n\t\t\tif currPath != entry.Path {\n\t\t\t\tchildEntry = client.StatInfo{\n\t\t\t\t\tIsPinned: false,\n\t\t\t\t\tPath: currPath,\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tchild = &treeNode{\n\t\t\t\tname: name,\n\t\t\t\tdepth: depth + 1,\n\t\t\t\tentry: childEntry,\n\t\t\t}\n\n\t\t\tchild.isLast = true\n\t\t\tif len(curr.order) > 0 {\n\t\t\t\tcurr.order[len(curr.order)-1].isLast = false\n\t\t\t}\n\n\t\t\tchild.parent = curr\n\t\t\tcurr.children[name] = child\n\t\t\tcurr.order = append(curr.order, child)\n\t\t}\n\n\t\tcurr = child\n\t}\n}\n\nfunc (n *treeNode) Len() int {\n\treturn len(n.order)\n}\n\nfunc (n *treeNode) Swap(i, j int) {\n\tn.order[i], n.order[j] = n.order[j], n.order[i]\n\n\t// This is not very clever, but works and is obvious:\n\tn.order[i].isLast = i == len(n.order)-1\n\tn.order[j].isLast = j == len(n.order)-1\n}\n\nfunc (n *treeNode) Less(i, j int) bool {\n\t// Sort case insensitive.\n\tiRunes := []rune(n.order[i].name)\n\tjRunes := []rune(n.order[j].name)\n\n\tfor idx := 0; idx < len(iRunes) && idx < len(jRunes); idx++ {\n\t\tir := iRunes[idx]\n\t\tjr := jRunes[idx]\n\n\t\tlir := unicode.ToLower(ir)\n\t\tljr := unicode.ToLower(jr)\n\n\t\tif lir != ljr {\n\t\t\treturn lir < ljr\n\t\t}\n\n\t\tif ir != jr {\n\t\t\treturn ir < jr\n\t\t}\n\t}\n\n\treturn false\n}\n\nfunc (n *treeNode) Print(cfg *treeCfg) {\n\tparents := make([]*treeNode, n.depth)\n\tcurr := n\n\n\tsort.Sort(n)\n\n\t// You could do probably go upwards and print to\n\t// a string buffer for performance, but this is probably\n\t// not necessary/critical here.\n\tfor i := 0; i < n.depth; i++ {\n\t\tparents[n.depth-i-1] = curr\n\t\tcurr = curr.parent\n\t}\n\n\tfor i := 0; i < n.depth; i++ {\n\t\tif i == n.depth-1 {\n\t\t\tif n.isLast {\n\t\t\t\tfmt.Printf(\"%s\", treeRuneCorner)\n\t\t\t} else {\n\t\t\t\tfmt.Printf(\"%s\", treeRuneTri)\n\t\t\t}\n\t\t} else {\n\t\t\tif parents[i].isLast {\n\t\t\t\tfmt.Printf(\"%s \", \" \")\n\t\t\t} else {\n\t\t\t\tfmt.Printf(\"%s \", treeRunePipe)\n\t\t\t}\n\t\t}\n\t}\n\n\t// Default to an auto-formatter:\n\tformat := cfg.format\n\tif format == nil {\n\t\tformat = func(n *treeNode) string {\n\t\t\tswitch {\n\t\t\tcase n.name == \"/\":\n\t\t\t\treturn color.MagentaString(\"•\")\n\t\t\tcase n.entry.IsDir:\n\t\t\t\treturn \" \" + color.GreenString(n.name+\"/\")\n\t\t\t}\n\n\t\t\treturn \" \" + n.name\n\t\t}\n\t}\n\n\tprefix := treeRuneBar\n\tif n.name == \"/\" {\n\t\tprefix = \"\"\n\t}\n\n\tformatted := format(n)\n\tpinState := \"\"\n\tif cfg.showPin && n.entry.IsPinned {\n\t\tpinState = \" \" + pinStateToSymbol(n.entry.IsPinned, n.entry.IsExplicit)\n\t}\n\n\tfmt.Printf(\"%s%s%s\\n\", prefix, formatted, pinState)\n\tfor _, child := range n.order {\n\t\tchild.Print(cfg)\n\t}\n}\n\nfunc showTree(entries []client.StatInfo, cfg *treeCfg) {\n\troot := &treeNode{name: \"/\"}\n\tnfiles, ndirs := 0, 0\n\n\thasRoot := false\n\tfor _, entry := range entries {\n\t\tif entry.Path == \"/\" {\n\t\t\troot.entry = entry\n\t\t\thasRoot = true\n\t\t} else {\n\t\t\troot.Insert(entry)\n\t\t}\n\n\t\tif entry.IsDir {\n\t\t\tndirs++\n\t\t} else {\n\t\t\tnfiles++\n\t\t}\n\t}\n\n\tif !hasRoot {\n\t\troot.entry = client.StatInfo{\n\t\t\tPath: \"/\",\n\t\t}\n\t}\n\n\troot.Print(cfg)\n\n\t// Speak understandable english:\n\tdirLabel := \"directories\"\n\tif ndirs == 1 {\n\t\tdirLabel = \"directory\"\n\t}\n\n\tfileLabel := \"files\"\n\tif nfiles == 1 {\n\t\tfileLabel = \"file\"\n\t}\n\n\tfmt.Printf(\"\\n%d %s, %d %s\\n\", ndirs, dirLabel, nfiles, fileLabel)\n}\n" }, { "path": "cmd/util.go", "content": "package cmd\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"math/rand\"\n\t\"net/url\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"regexp\"\n\t\"strconv\"\n\t\"strings\"\n\t\"text/template\"\n\t\"time\"\n\n\t\"github.com/fatih/color\"\n\t\"github.com/sahib/brig/client\"\n\t\"github.com/sahib/brig/defaults\"\n\t\"github.com/sahib/brig/util\"\n\t\"github.com/sahib/config\"\n\tlog \"github.com/sirupsen/logrus\"\n\t\"github.com/urfave/cli\"\n)\n\nvar (\n\t// backend delivers overly descriptive error messages including\n\t// the string below. Simply filter this info:\n\trpcErrPattern = regexp.MustCompile(`\\s*server/capnp/local_api.capnp.*rpc exception:\\s*`)\n)\n\n// ExitCode is an error that maps the error interface to a specific error\n// message and a unix exit code\ntype ExitCode struct {\n\tCode int\n\tMessage string\n}\n\nfunc (err ExitCode) Error() string {\n\treturn err.Message\n}\n\nfunc mustAbsPath(path string) string {\n\tabsPath, err := filepath.Abs(path)\n\tif err != nil {\n\t\tfmt.Printf(\"Failed to get absolute repo path: %v\", err)\n\t\tos.Exit(1)\n\t}\n\n\treturn absPath\n}\n\nfunc yesify(val bool) string {\n\tif val {\n\t\treturn color.GreenString(\"yes\")\n\t}\n\n\treturn color.RedString(\"no\")\n}\n\nfunc checkmarkify(val bool) string {\n\tif val {\n\t\treturn color.GreenString(\"✔\")\n\t}\n\n\treturn \"\"\n}\n\n// guessRepoFolder tries to find the repository path by using a number of\n// sources. This helper may call exit when it fails to get the path.\nfunc guessRepoFolder(ctx *cli.Context) (string, error) {\n\tif ctx.GlobalIsSet(\"repo\") {\n\t\t// No guessing needed, follow user wish.\n\t\treturn ctx.GlobalString(\"repo\"), nil\n\t}\n\n\tguessLocations := []string{\n\t\t// TODO: For now just one.\n\t\t\".\",\n\t}\n\n\tvar lastError error\n\tfor _, guessLocation := range guessLocations {\n\t\trepoFolder := mustAbsPath(guessLocation)\n\t\tif _, err := os.Stat(filepath.Join(repoFolder, \"config.yml\")); err != nil {\n\t\t\tlastError = err\n\t\t\tcontinue\n\t\t}\n\n\t\treturn repoFolder, nil\n\t}\n\n\treturn \"\", lastError\n}\n\nfunc openConfig(folder string) (*config.Config, error) {\n\tconfigPath := filepath.Join(folder, \"config.yml\")\n\tcfg, err := defaults.OpenMigratedConfig(configPath)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"could not find config: %v\", err)\n\t}\n\n\treturn cfg, nil\n}\n\nfunc guessDaemonURL(ctx *cli.Context) (string, error) {\n\tif ctx.GlobalIsSet(\"url\") {\n\t\t// No guessing needed, follow user wish.\n\t\treturn ctx.GlobalString(\"url\"), nil\n\t}\n\n\tfolder, err := guessRepoFolder(ctx)\n\tif err != nil {\n\t\tlog.Warnf(\"note: I don't know where the repository is or cannot read it.\")\n\t\tlog.Warnf(\" I will continue with default values, cross fingers.\")\n\t\tlog.Warnf(\" We recommend to set BRIG_PATH or pass --repo always.\")\n\t\tlog.Warnf(\" Alternatively you can cd to your repository.\")\n\t\treturn ctx.GlobalString(\"url\"), err\n\t}\n\n\tcfg, err := openConfig(folder)\n\tif err != nil {\n\t\t// Assume default:\n\t\treturn ctx.GlobalString(\"url\"), nil\n\t}\n\n\treturn cfg.String(\"daemon.url\"), nil\n}\n\nfunc guessFreeDaemonURL(ctx *cli.Context, owner string) (string, error) {\n\tif ctx.GlobalIsSet(\"url\") {\n\t\t// No guessing needed, follow user wish.\n\t\treturn ctx.GlobalString(\"url\"), nil\n\t}\n\n\tdefaultURL := defaults.DaemonDefaultURL()\n\tu, err := url.Parse(defaultURL)\n\tif err != nil {\n\t\t// this is a programming error\n\t\tpanic(\"invalid hard-coded default daemon url\")\n\t}\n\n\tswitch u.Scheme {\n\tcase \"unix\":\n\t\t// Distinguish the path, so that we can have\n\t\t// several daemon on a single system.\n\t\tv := u.Query()\n\t\tv.Add(\"id\", strings.ReplaceAll(owner, \"/\", \"_\"))\n\t\tu.RawQuery = v.Encode()\n\t\treturn u.String(), nil\n\tcase \"tcp\":\n\t\t// Do a best effort by searching for a free port\n\t\t// and use that for the brig repository.\n\t\t// This might be racy, but at least try it.\n\t\tport := util.FindFreePort()\n\t\treturn fmt.Sprintf(\"tcp://127.0.0.1:%d\", port), nil\n\tdefault:\n\t\treturn \"\", fmt.Errorf(\"default url has unknown \")\n\t}\n}\n\nfunc prefixSlash(s string) string {\n\tif !strings.HasPrefix(s, \"/\") {\n\t\treturn \"/\" + s\n\t}\n\n\treturn s\n}\n\ntype cmdHandlerWithClient func(ctx *cli.Context, ctl *client.Client) error\n\nfunc getExecutablePath() (string, error) {\n\t// NOTE: This might not work on other platforms.\n\t// In this case we fall back to LookPath().\n\texePath, err := os.Readlink(\"/proc/self/exe\")\n\tif err != nil {\n\t\treturn exec.LookPath(\"brig\")\n\t}\n\n\treturn filepath.Clean(exePath), nil\n}\n\nfunc startDaemon(ctx *cli.Context, repoPath, daemonURL string) (*client.Client, error) {\n\tstat, err := os.Stat(repoPath)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif !stat.IsDir() {\n\t\treturn nil, fmt.Errorf(\"»%s« is not a directory\", repoPath)\n\t}\n\n\texePath, err := getExecutablePath()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tlogVerbose(ctx, \"using executable path: %s\", exePath)\n\n\tlogVerbose(\n\t\tctx,\n\t\t\"No Daemon running at %s. Starting daemon from binary: %s\",\n\t\tdaemonURL,\n\t\texePath,\n\t)\n\n\tdaemonArgs := []string{\n\t\t\"--repo\", repoPath,\n\t\t\"--url\", daemonURL,\n\t\t\"daemon\", \"launch\",\n\t}\n\n\targString := fmt.Sprintf(\"'%s'\", strings.Join(daemonArgs, \"' '\"))\n\tlogVerbose(ctx, \"Starting daemon as: %s %s\", exePath, argString)\n\n\tproc := exec.Command(exePath, daemonArgs...) // #nosec\n\tproc.Env = append(proc.Env, fmt.Sprintf(\"PATH=%s\", os.Getenv(\"PATH\")))\n\tif err := proc.Start(); err != nil {\n\t\tlog.Infof(\"Failed to start the daemon: %v\", err)\n\t\treturn nil, err\n\t}\n\n\t// This will likely suffice for most cases:\n\ttime.Sleep(500 * time.Millisecond)\n\n\twarningPrinted := false\n\tfor i := 0; i < 500; i++ {\n\t\tctl, err := client.Dial(context.Background(), daemonURL)\n\t\tif err != nil {\n\t\t\t// Only print this warning once...\n\t\t\tif !warningPrinted && i >= 100 {\n\t\t\t\tlog.Warnf(\"waiting a bit long for daemon to bootup...\")\n\t\t\t\twarningPrinted = true\n\t\t\t}\n\t\t\ttime.Sleep(50 * time.Millisecond)\n\t\t\tcontinue\n\t\t}\n\n\t\treturn ctl, nil\n\t}\n\n\treturn nil, fmt.Errorf(\"Daemon could not be started or took to long\")\n}\n\nfunc isDaemonRunning(ctx *cli.Context) (bool, error) {\n\tdaemonURL, err := guessDaemonURL(ctx)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\ttctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)\n\tdefer cancel()\n\n\tctl, err := client.Dial(tctx, daemonURL)\n\tif err != nil {\n\t\treturn false, nil\n\t}\n\n\tdefer ctl.Close()\n\treturn true, nil\n}\n\nfunc withDaemon(handler cmdHandlerWithClient, startNew bool) cli.ActionFunc {\n\treturn func(ctx *cli.Context) error {\n\t\tdaemonURL, _ := guessDaemonURL(ctx)\n\n\t\tif startNew {\n\t\t\tlogVerbose(ctx, \"Using url %s to check for running daemon.\", daemonURL)\n\t\t} else {\n\t\t\tlogVerbose(ctx, \"Using url %s to connect to existing daemon.\", daemonURL)\n\t\t}\n\n\t\t// Check if the daemon is running already:\n\t\tctl, err := client.Dial(context.Background(), daemonURL)\n\t\tif err == nil {\n\t\t\tdefer ctl.Close()\n\t\t\treturn handler(ctx, ctl)\n\t\t}\n\n\t\tif !startNew {\n\t\t\t// Daemon was not running and we may not start a new one.\n\t\t\treturn ExitCode{DaemonNotResponding, \"Daemon not running\"}\n\t\t}\n\n\t\t// Start the server & pass the password:\n\t\tfolder, err := guessRepoFolder(ctx)\n\t\tif err != nil {\n\t\t\treturn ExitCode{\n\t\t\t\tBadArgs,\n\t\t\t\tfmt.Sprintf(\"could not guess folder: %v\", err),\n\t\t\t}\n\t\t}\n\n\t\tlogVerbose(ctx, \"starting new daemon in background, on folder '%s'\", folder)\n\n\t\tctl, err = startDaemon(ctx, folder, daemonURL)\n\t\tif err != nil {\n\t\t\treturn ExitCode{\n\t\t\t\tDaemonNotResponding,\n\t\t\t\tfmt.Sprintf(\"Unable to start daemon: %v\", err),\n\t\t\t}\n\t\t}\n\n\t\t// Run the actual handler:\n\t\tdefer ctl.Close()\n\t\treturn handler(ctx, ctl)\n\t}\n}\n\ntype checkFunc func(ctx *cli.Context) int\n\nfunc withArgCheck(checker checkFunc, handler cli.ActionFunc) cli.ActionFunc {\n\treturn func(ctx *cli.Context) error {\n\t\tif checker(ctx) != Success {\n\t\t\tos.Exit(BadArgs)\n\t\t}\n\n\t\treturn handler(ctx)\n\t}\n}\n\nfunc prettyPrintError(err error) string {\n\treturn rpcErrPattern.ReplaceAllString(err.Error(), \" \")\n}\n\nfunc needAtLeast(min int) checkFunc {\n\treturn func(ctx *cli.Context) int {\n\t\tif ctx.NArg() < min {\n\t\t\tif min == 1 {\n\t\t\t\tlog.Warningf(\"Need at least %d argument.\", min)\n\t\t\t} else {\n\t\t\t\tlog.Warningf(\"Need at least %d arguments.\", min)\n\t\t\t}\n\n\t\t\tif err := cli.ShowCommandHelp(ctx, ctx.Command.Name); err != nil {\n\t\t\t\tlog.Warningf(\"Failed to display --help: %v\", err)\n\t\t\t}\n\n\t\t\treturn BadArgs\n\t\t}\n\n\t\treturn Success\n\t}\n}\n\nfunc isNonEmptyDir(dir string) (bool, error) {\n\tfd, err := os.Open(dir) // #nosec\n\tif err != nil && os.IsNotExist(err) {\n\t\treturn false, nil\n\t}\n\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\tnames, err := fd.Readdirnames(-1)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\t// dumb heuristic: if there's stuff in there,\n\t// assume we shouldn't init over.\n\treturn len(names) >= 1, nil\n}\n\n// tempFileWithSuffix works the same as ioutil.TempFile(),\n// but allows for the addition of a suffix to the filepath.\n// This has the nice side effect that some editors can recognize\n// the filetype based on the ending and provide you syntax highlighting.\n// (this is used in edit() below)\nfunc tempFileWithSuffix(dir, prefix, suffix string) (f *os.File, err error) {\n\tif dir == \"\" {\n\t\tdir = os.TempDir()\n\t}\n\n\tfor i := 0; i < 10000; i++ {\n\t\tmid := strconv.Itoa(rand.Int()) // #nosec\n\t\tname := filepath.Join(dir, prefix+mid+suffix)\n\t\tf, err = os.OpenFile(name, os.O_RDWR|os.O_CREATE|os.O_EXCL, 0600)\n\t\tif os.IsExist(err) {\n\t\t\tcontinue\n\t\t}\n\t\tbreak\n\t}\n\treturn\n}\n\n// editToPath opens up $EDITOR with `data` and saves the edited data\n// to a temporary path that is then returned.\nfunc editToPath(data []byte, suffix string) (string, error) {\n\teditor := os.Getenv(\"EDITOR\")\n\tif editor == \"\" {\n\t\t// It makes my heart bleed, but assume that vi is too hard\n\t\t// for the majority I've met & that might use brig.\n\t\teditor = \"nano\"\n\t}\n\n\tfd, err := tempFileWithSuffix(\"\", \"brig-cmd-buffer-\", suffix)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\tdoDelete := false\n\n\t// Make sure it gets cleaned up.\n\tdefer func() {\n\t\tif doDelete {\n\t\t\tif err := os.Remove(fd.Name()); err != nil {\n\t\t\t\tfmt.Printf(\"Failed to remove temp file: %v\\n\", err)\n\t\t\t}\n\t\t}\n\n\t\tif err := fd.Close(); err != nil {\n\t\t\tfmt.Printf(\"Failed to close file: %v\\n\", err)\n\t\t}\n\t}()\n\n\tif _, err := fd.Write(data); err != nil {\n\t\treturn \"\", err\n\t}\n\n\t// Launch editor and hook it up with all necessary fds:\n\tcmd := exec.Command(editor, fd.Name()) // #nosec\n\tcmd.Stdin = os.Stdin\n\tcmd.Stdout = os.Stdout\n\tcmd.Stderr = os.Stderr\n\n\tif err := cmd.Run(); err != nil {\n\t\tdoDelete = true\n\t\treturn \"\", fmt.Errorf(\"Running $EDITOR (%s) failed: %v\", editor, err)\n\t}\n\n\tif _, err := fd.Seek(0, io.SeekStart); err != nil {\n\t\tdoDelete = true\n\t\treturn \"\", err\n\t}\n\n\treturn fd.Name(), nil\n}\n\n// edit opens up $EDITOR with `data` and returns the edited data.\nfunc edit(data []byte, suffix string) ([]byte, error) {\n\ttempPath, err := editToPath(data, suffix)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdefer func() {\n\t\tif err := os.Remove(tempPath); err != nil {\n\t\t\tfmt.Printf(\"Failed to remove temp file: %v\\n\", err)\n\t\t}\n\t}()\n\n\tnewData, err := ioutil.ReadFile(tempPath) // #nosec\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Some editors might add a trailing newline:\n\treturn bytes.TrimRight(newData, \"\\n\"), nil\n}\n\n// parseDuration tries to convert the string `s` to\n// a duration in seconds (+ fractions).\n// It uses time.ParseDuration() internally, but allows\n// whole numbers which are counted as seconds.\nfunc parseDuration(s string) (float64, error) {\n\tsec, err := strconv.ParseFloat(s, 64)\n\tif err == nil {\n\t\treturn sec, nil\n\t}\n\n\tdur, err := time.ParseDuration(s)\n\tif err != nil {\n\t\treturn 0.0, err\n\t}\n\n\treturn float64(dur) / float64(time.Second), nil\n}\n\nfunc readFormatTemplate(ctx *cli.Context) (*template.Template, error) {\n\tif ctx.IsSet(\"format\") {\n\t\tsource := ctx.String(\"format\") + \"\\n\"\n\t\ttmpl, err := template.New(\"format\").Parse(source)\n\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn tmpl, nil\n\t}\n\n\treturn nil, nil\n}\n\nfunc pinStateToSymbol(isPinned, isExplicit bool) string {\n\tif isPinned {\n\t\tcolorFn := color.CyanString\n\t\tif isExplicit {\n\t\t\tcolorFn = color.MagentaString\n\t\t}\n\n\t\treturn colorFn(\"✔\")\n\t}\n\n\treturn \"\"\n}\n\nfunc yesOrNo(v bool) string {\n\tif v {\n\t\treturn color.GreenString(\"yes\")\n\t}\n\n\treturn color.RedString(\"no\")\n}\n\ntype logWriter struct{ prefix string }\n\nfunc (lw *logWriter) Write(buf []byte) (int, error) {\n\tlog.Infof(\"%s: %s\", lw.prefix, string(bytes.TrimSpace(buf)))\n\treturn len(buf), nil\n}\n" }, { "path": "cmd/vcs_handlers.go", "content": "package cmd\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"path\"\n\t\"path/filepath\"\n\t\"sort\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/sahib/brig/cmd/tabwriter\"\n\n\t\"github.com/fatih/color\"\n\t\"github.com/sahib/brig/client\"\n\t\"github.com/urfave/cli\"\n)\n\nfunc handleReset(ctx *cli.Context, ctl *client.Client) error {\n\tforce := ctx.Bool(\"force\")\n\trev := ctx.Args().First()\n\tpath := \"\"\n\n\tif len(ctx.Args()) > 1 {\n\t\tpath = ctx.Args().Get(1)\n\t}\n\n\tif err := ctl.Reset(path, rev, force); err != nil {\n\t\treturn ExitCode{UnknownError, fmt.Sprintf(\"reset: %v\", err)}\n\t}\n\n\treturn nil\n}\n\nfunc commitName(cmt *client.Commit) string {\n\tif cmt == nil {\n\t\treturn \"\"\n\t}\n\n\tif len(cmt.Tags) > 0 {\n\t\treturn strings.ToUpper(cmt.Tags[0])\n\t}\n\n\treturn cmt.Hash.ShortB58()\n}\n\nfunc handleHistory(ctx *cli.Context, ctl *client.Client) error {\n\tpath := ctx.Args().First()\n\n\thistory, err := ctl.History(path)\n\tif err != nil {\n\t\treturn ExitCode{UnknownError, fmt.Sprintf(\"history: %v\", err)}\n\t}\n\n\tif _, err := ctl.Stat(path); err != nil {\n\t\tfmt.Printf(\"%s %s\",\n\t\t\tcolor.YellowString(\"WARNING:\"),\n\t\t\t`This file is not part of this commit, but there's still history for it.\n Most likely this file was moved or removed in the past.\n\n`)\n\t}\n\n\ttabW := tabwriter.NewWriter(\n\t\tos.Stdout, 0, 0, 2, ' ',\n\t\ttabwriter.StripEscape,\n\t)\n\n\tcontainsMoves := false\n\tfor _, entry := range history {\n\t\tfor _, detail := range entry.Mask {\n\t\t\tif detail == \"moved\" {\n\t\t\t\tcontainsMoves = true\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\n\t\tif containsMoves {\n\t\t\tbreak\n\t\t}\n\t}\n\n\tif len(history) != 0 {\n\t\tif containsMoves {\n\t\t\tfmt.Fprintf(tabW, \"CHANGE\\tFROM\\tTO\\tHOW\\tWHEN\\tPIN\\t\\n\")\n\t\t} else {\n\t\t\tfmt.Fprintf(tabW, \"CHANGE\\tFROM\\tTO\\t\\tWHEN\\tPIN\\t\\n\")\n\t\t}\n\t}\n\n\tfor _, entry := range history {\n\t\twhat := \"\"\n\t\tprintLine := true\n\n\t\tfor _, detail := range entry.Mask {\n\t\t\t// If it was moved, let's display what moved.\n\t\t\tif detail == \"moved\" {\n\t\t\t\tsrc := entry.WasPreviouslyAt\n\t\t\t\tdst := entry.Path\n\n\t\t\t\tif entry.MovedTo != \"\" {\n\t\t\t\t\tdst = entry.MovedTo\n\t\t\t\t}\n\n\t\t\t\twhat = fmt.Sprintf(\n\t\t\t\t\t\"%s → %s\", color.RedString(src), color.RedString(dst),\n\t\t\t\t)\n\t\t\t}\n\n\t\t\t// Only display empty changes if nothing happened.\n\t\t\tif detail == \"none\" && !ctx.Bool(\"empty\") {\n\t\t\t\tprintLine = false\n\t\t\t}\n\t\t}\n\n\t\tif !printLine {\n\t\t\tcontinue\n\t\t}\n\n\t\tchangeDesc := color.YellowString(strings.Join(entry.Mask, \", \"))\n\t\twhen := color.MagentaString(entry.Head.Date.Format(time.UnixDate))\n\n\t\tfmt.Fprintf(\n\t\t\ttabW,\n\t\t\t\"%s\\t%s\\t%s\\t%s\\t%s\\t%s\\t%s\\n\",\n\t\t\tchangeDesc,\n\t\t\tcolor.CyanString(commitName(entry.Next)),\n\t\t\tcolor.GreenString(commitName(entry.Head)),\n\t\t\twhat,\n\t\t\twhen,\n\t\t\tpinStateToSymbol(entry.IsPinned, entry.IsExplicit),\n\t\t\tentry.Head.Msg,\n\t\t)\n\t}\n\n\treturn tabW.Flush()\n}\n\n// makePathAbbrev tries to abbreviate the `dst` path if\n// both are in the same directory.\nfunc makePathAbbrev(srcNd, dstNd client.StatInfo) string {\n\tsrc, dst := srcNd.Path, dstNd.Path\n\tif path.Dir(src) == path.Dir(dst) {\n\t\tdstBase := path.Base(dst)\n\t\tif dstNd.IsDir {\n\t\t\treturn dstBase + \"/\"\n\n\t\t}\n\n\t\treturn dstBase\n\t}\n\n\trelPath, err := filepath.Rel(path.Dir(src), dst)\n\tif err != nil {\n\t\tfmt.Println(\"Failed to get relative path: \", err)\n\t\tif dstNd.IsDir {\n\t\t\treturn dst + \"/\"\n\t\t}\n\n\t\treturn dst\n\t}\n\n\t// We could also possibly check here if relPath is longer than dst\n\t// and only display the relative version then. But being consistent\n\t// is more valuable here I think.\n\tif dstNd.IsDir {\n\t\treturn relPath + \"/\"\n\t}\n\n\treturn relPath\n}\n\nfunc suffixIfDir(nd *treeNode) string {\n\tif nd.entry.IsDir {\n\t\treturn nd.name + \"/\"\n\t}\n\n\treturn nd.name\n}\n\nconst (\n\tdiffTypeNone = iota\n\tdiffTypeAdded\n\tdiffTypeRemoved\n\tdiffTypeMissing\n\tdiffTypeMoved\n\tdiffTypeIgnored\n\tdiffTypeConflict\n\tdiffTypeMerged\n)\n\ntype diffEntry struct {\n\ttyp int\n\tpair client.DiffPair\n}\n\n// Called to format each name in the resulting tree:\nfunc printDiffTreeLineFormatter(types map[string]diffEntry, n *treeNode) string {\n\tif n.name == \"/\" {\n\t\treturn color.MagentaString(\"•\")\n\t}\n\n\tif diffEntry, ok := types[n.entry.Path]; ok {\n\t\tswitch diffEntry.typ {\n\t\tcase diffTypeAdded:\n\t\t\treturn color.GreenString(\" + \" + \"▩ ← \" + suffixIfDir(n))\n\t\tcase diffTypeRemoved:\n\t\t\treturn color.RedString(\" - \" + suffixIfDir(n) + \" ← ▩\")\n\t\tcase diffTypeMissing:\n\t\t\treturn color.MagentaString(\" _ \" + suffixIfDir(n) + \" → ▩\")\n\t\tcase diffTypeIgnored:\n\t\t\treturn color.YellowString(\" * \" + suffixIfDir(n))\n\t\tcase diffTypeMoved:\n\t\t\tsrcPath := makePathAbbrev(diffEntry.pair.Dst, diffEntry.pair.Src)\n\t\t\tdstBase := path.Base(diffEntry.pair.Dst.Path)\n\t\t\tif diffEntry.pair.Src.IsDir {\n\t\t\t\tdstBase += \"/\"\n\t\t\t}\n\n\t\t\treturn color.CyanString(fmt.Sprintf(\" %s ↔ %s\", dstBase, srcPath))\n\t\tcase diffTypeMerged:\n\t\t\tdstPath := makePathAbbrev(diffEntry.pair.Dst, diffEntry.pair.Src)\n\t\t\tsrcBase := path.Base(diffEntry.pair.Src.Path)\n\t\t\tif diffEntry.pair.Src.IsDir {\n\t\t\t\tsrcBase += \"/\"\n\t\t\t}\n\t\t\t// Attempt to figure out which way merge should go\n\t\t\t// based on modification times.\n\t\t\t// This information was available at resolver time in the PairDiff\n\t\t\t// but server returns simplified PairDiff without modification masks.\n\t\t\tsrcModTime := diffEntry.pair.Src.ModTime\n\t\t\tdstModTime := diffEntry.pair.Dst.ModTime\n\t\t\tvar mergeSymbol string = color.MagentaString(\"→\")\n\t\t\tif srcModTime.After(dstModTime) {\n\t\t\t\tmergeSymbol = color.GreenString(\"←\")\n\t\t\t}\n\n\t\t\treturn color.WhiteString(fmt.Sprintf(\" %s %s %s \", dstPath, mergeSymbol, srcBase))\n\t\tcase diffTypeConflict:\n\t\t\tdstPath := makePathAbbrev(diffEntry.pair.Dst, diffEntry.pair.Src)\n\t\t\tsrcBase := path.Base(diffEntry.pair.Src.Path)\n\t\t\tif diffEntry.pair.Src.IsDir {\n\t\t\t\tsrcBase += \"/\"\n\t\t\t}\n\n\t\t\treturn color.MagentaString(fmt.Sprintf(\" %s ⚡%s\", dstPath, srcBase))\n\t\t}\n\t}\n\n\treturn n.name\n}\n\nfunc printDiffTree(diff *client.Diff, printMissing bool) {\n\tentries := []client.StatInfo{}\n\ttypes := make(map[string]diffEntry)\n\n\t// Singular types:\n\tfor _, info := range diff.Added {\n\t\ttypes[info.Path] = diffEntry{typ: diffTypeAdded}\n\t\tentries = append(entries, info)\n\t}\n\tfor _, info := range diff.Removed {\n\t\ttypes[info.Path] = diffEntry{typ: diffTypeRemoved}\n\t\tentries = append(entries, info)\n\t}\n\n\tif printMissing {\n\t\tfor _, info := range diff.Missing {\n\t\t\ttypes[info.Path] = diffEntry{typ: diffTypeMissing}\n\t\t\tentries = append(entries, info)\n\t\t}\n\t}\n\n\tfor _, info := range diff.Ignored {\n\t\ttypes[info.Path] = diffEntry{typ: diffTypeIgnored}\n\t\tentries = append(entries, info)\n\t}\n\n\t// Pair types:\n\tfor _, pair := range diff.Moved {\n\t\ttypes[pair.Dst.Path] = diffEntry{\n\t\t\ttyp: diffTypeMoved,\n\t\t\tpair: pair,\n\t\t}\n\t\tentries = append(entries, pair.Dst)\n\t}\n\tfor _, pair := range diff.Conflict {\n\t\ttypes[pair.Dst.Path] = diffEntry{\n\t\t\ttyp: diffTypeConflict,\n\t\t\tpair: pair,\n\t\t}\n\t\tentries = append(entries, pair.Dst)\n\t}\n\tfor _, pair := range diff.Merged {\n\t\ttypes[pair.Dst.Path] = diffEntry{\n\t\t\ttyp: diffTypeMerged,\n\t\t\tpair: pair,\n\t\t}\n\t\tentries = append(entries, pair.Dst)\n\t}\n\n\tif len(entries) == 0 {\n\t\t// Nothing to show:\n\t\treturn\n\t}\n\n\tsort.Slice(entries, func(i, j int) bool {\n\t\treturn entries[i].Path < entries[j].Path\n\t})\n\n\t// Render the tree:\n\tshowTree(entries, &treeCfg{\n\t\tformat: func(n *treeNode) string {\n\t\t\treturn printDiffTreeLineFormatter(types, n)\n\t\t},\n\t\tshowPin: false,\n\t})\n}\n\nfunc isEmptyDiff(diff *client.Diff) bool {\n\treturn 0 == 0+\n\t\tlen(diff.Added)+\n\t\tlen(diff.Conflict)+\n\t\tlen(diff.Ignored)+\n\t\tlen(diff.Merged)+\n\t\tlen(diff.Missing)+\n\t\tlen(diff.Moved)+\n\t\tlen(diff.Removed)\n}\n\nfunc printDiff(diff *client.Diff, printMissing bool) {\n\tsimpleSection := func(heading string, infos []client.StatInfo) {\n\t\tif len(infos) == 0 {\n\t\t\treturn\n\t\t}\n\n\t\tfmt.Println(heading)\n\t\tfor _, info := range infos {\n\t\t\tpath := info.Path\n\t\t\tif info.IsDir {\n\t\t\t\tpath += \"/\"\n\t\t\t}\n\n\t\t\tfmt.Printf(\" %s\\n\", path)\n\t\t}\n\n\t\tfmt.Println()\n\t}\n\n\tpairSection := func(heading, symbol string, infos []client.DiffPair) {\n\t\tif len(infos) == 0 {\n\t\t\treturn\n\t\t}\n\n\t\tfmt.Println(heading)\n\t\tfor _, pair := range infos {\n\t\t\tsrcPath := pair.Src.Path\n\t\t\tif pair.Src.IsDir {\n\t\t\t\tsrcPath += \"/\"\n\t\t\t}\n\n\t\t\tdstPath := pair.Dst.Path\n\t\t\tif pair.Dst.IsDir {\n\t\t\t\tdstPath += \"/\"\n\t\t\t}\n\n\t\t\tif pair.Src.Path != pair.Dst.Path {\n\t\t\t\tfmt.Printf(\" %s %s %s\\n\", dstPath, symbol, srcPath)\n\t\t\t} else {\n\t\t\t\tfmt.Printf(\" %s %s\\n\", symbol, srcPath)\n\t\t\t}\n\t\t}\n\n\t\tfmt.Println()\n\t}\n\n\tvar addedAtRemote []client.DiffPair\n\tfor _, src := range diff.Added {\n\t\tvar pair client.DiffPair\n\t\tpair.Dst.Path = \"▩\"\n\t\tpair.Src = src\n\t\taddedAtRemote = append(addedAtRemote, pair)\n\t}\n\tpairSection(color.GreenString(\"Added:\"), \"←\", addedAtRemote)\n\n\tsimpleSection(color.YellowString(\"Ignored:\"), diff.Ignored)\n\n\tvar removedAtRemote []client.DiffPair\n\tfor _, dst := range diff.Removed {\n\t\tvar pair client.DiffPair\n\t\tpair.Dst = dst\n\t\tpair.Src.Path = \"▩\"\n\t\tremovedAtRemote = append(removedAtRemote, pair)\n\t}\n\tpairSection(color.RedString(\"Removed:\"), \"←\", removedAtRemote)\n\n\t// split diff.Merged to changedLocally and changedRemotely arrays\n\tvar changedLocally, changedRemotely []client.DiffPair\n\tfor _, pair := range diff.Merged {\n\t\tsrcModTime := pair.Src.ModTime\n\t\tdstModTime := pair.Dst.ModTime\n\t\tif srcModTime.After(dstModTime) {\n\t\t\tchangedRemotely = append(changedRemotely, pair)\n\t\t} else {\n\t\t\tchangedLocally = append(changedLocally, pair)\n\t\t}\n\t}\n\n\tif printMissing {\n\t\tvar missedAtRemote []client.DiffPair\n\t\tfor _, dst := range diff.Missing {\n\t\t\tvar pair client.DiffPair\n\t\t\tpair.Dst = dst\n\t\t\tpair.Src.Path = \"▩\"\n\t\t\tmissedAtRemote = append(missedAtRemote, pair)\n\t\t}\n\t\tpairSection(color.RedString(\"Missing:\"), \"→\", missedAtRemote)\n\t}\n\n\tpairSection(color.CyanString(\"Moved:\"), \"↔\", diff.Moved)\n\tpairSection(color.WhiteString(\"Changed Locally:\"), \"→\", changedLocally)\n\tpairSection(color.WhiteString(\"Changed Remotely:\"), \"←\", changedRemotely)\n\tpairSection(color.MagentaString(\"Conflicts:\"), \"⚡\", diff.Conflict)\n}\n\nfunc handleDiff(ctx *cli.Context, ctl *client.Client) error {\n\tif ctx.NArg() > 4 {\n\t\tfmt.Println(\"More than four arguments can't be handled.\")\n\t}\n\n\tself, err := ctl.Whoami()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tlocalName := self.CurrentUser\n\tremoteName := self.CurrentUser\n\n\tremoteRev := \"CURR\"\n\tlocalRev := \"CURR\"\n\n\tnArgs := ctx.NArg()\n\tif nArgs == 0 {\n\t\t// Special case: When typing brig diff we want to show\n\t\t// the diff from our CURR to HEAD only.\n\t\tlocalRev = \"HEAD\"\n\t}\n\n\tif ctx.Bool(\"self\") {\n\t\tswitch {\n\t\tcase nArgs >= 2:\n\t\t\tlocalRev = ctx.Args().Get(1)\n\t\t\tfallthrough\n\t\tcase nArgs >= 1:\n\t\t\tremoteRev = ctx.Args().Get(0)\n\t\t}\n\t} else {\n\t\tswitch {\n\t\tcase nArgs >= 4:\n\t\t\tlocalRev = ctx.Args().Get(3)\n\t\t\tfallthrough\n\t\tcase nArgs >= 3:\n\t\t\tremoteRev = ctx.Args().Get(2)\n\t\t\tfallthrough\n\t\tcase nArgs >= 2:\n\t\t\tlocalName = ctx.Args().Get(1)\n\t\t\tfallthrough\n\t\tcase nArgs >= 1:\n\t\t\tremoteName = ctx.Args().Get(0)\n\t\t}\n\t}\n\n\tneedFetch := !ctx.Bool(\"offline\")\n\tdiff, err := ctl.MakeDiff(localName, remoteName, localRev, remoteRev, needFetch)\n\tif err != nil {\n\t\treturn ExitCode{UnknownError, fmt.Sprintf(\"diff: %v\", err)}\n\t}\n\n\tprintMissing := ctx.Bool(\"missing\")\n\tif ctx.Bool(\"list\") {\n\t\tprintDiff(diff, printMissing)\n\t} else {\n\t\tprintDiffTree(diff, printMissing)\n\t}\n\n\treturn nil\n}\n\nfunc handleFetch(ctx *cli.Context, ctl *client.Client) error {\n\twho := ctx.Args().First()\n\treturn ctl.Fetch(who)\n}\n\nfunc handleSync(ctx *cli.Context, ctl *client.Client) error {\n\tif len(ctx.Args()) > 0 {\n\t\treturn handleSyncSingle(ctx, ctl, ctx.Args().First())\n\t}\n\n\tremotes, err := ctl.RemoteLs()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor _, rmt := range remotes {\n\t\t_, err := ctl.RemotePing(rmt.Name)\n\t\tif err != nil {\n\t\t\tfmt.Printf(\"Cannot reach %s..\\n\", rmt.Name)\n\t\t\tcontinue\n\t\t}\n\n\t\tfmt.Printf(\"Syncing with `%s`...\\n\", rmt.Name)\n\t\tif err := handleSyncSingle(ctx, ctl, rmt.Name); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc handleSyncSingle(ctx *cli.Context, ctl *client.Client, remoteName string) error {\n\tneedFetch := true\n\tif ctx.Bool(\"no-fetch\") {\n\t\tneedFetch = false\n\t}\n\n\tif ctx.Bool(\"quiet\") {\n\t\treturn nil\n\t}\n\n\tdiff, err := ctl.Sync(remoteName, needFetch)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif isEmptyDiff(diff) {\n\t\tfmt.Println(\"Nothing changed.\")\n\t\treturn nil\n\t}\n\n\tprintDiff(diff, false)\n\treturn nil\n}\n\nfunc handleStatus(ctx *cli.Context, ctl *client.Client) error {\n\tself, err := ctl.Whoami()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tcurr := self.CurrentUser\n\tdiff, err := ctl.MakeDiff(curr, curr, \"HEAD\", \"CURR\", false)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif ctx.Bool(\"tree\") {\n\t\tprintDiffTree(diff, false)\n\t} else {\n\t\tprintDiff(diff, false)\n\t}\n\n\treturn nil\n}\n\nfunc handleBecome(ctx *cli.Context, ctl *client.Client) error {\n\tbecomeSelf := ctx.Bool(\"self\")\n\tif !becomeSelf && ctx.NArg() < 1 {\n\t\treturn fmt.Errorf(\"become needs at least one argument without -s\")\n\t}\n\n\twhoami, err := ctl.Whoami()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\twho := ctx.Args().First()\n\tif becomeSelf {\n\t\twho = whoami.Owner\n\t}\n\n\tif whoami.CurrentUser == who {\n\t\tfmt.Printf(\"You are already %s.\\n\", color.GreenString(who))\n\t\treturn nil\n\t}\n\n\tif err := ctl.Become(who); err != nil {\n\t\treturn err\n\t}\n\n\tsuffix := \"Everything is read only.\"\n\tif who == whoami.Owner {\n\t\tsuffix = \"Welcome back!\"\n\t}\n\n\tfmt.Printf(\n\t\t\"You are viewing %s's data now. %s\\n\",\n\t\tcolor.GreenString(who), suffix,\n\t)\n\treturn nil\n}\n\nfunc handleCommit(ctx *cli.Context, ctl *client.Client) error {\n\tvar msg string\n\n\t// Build the message:\n\tif ctx.IsSet(\"message\") {\n\t\tmsg = ctx.String(\"message\")\n\t} else if len(ctx.Args()) >= 1 {\n\t\tmsg = strings.Join(ctx.Args(), \" \")\n\t} else {\n\t\tmsg = \"manual commit\"\n\t}\n\n\t// Send the commit:\n\tif err := ctl.MakeCommit(msg); err != nil {\n\t\treturn ExitCode{UnknownError, fmt.Sprintf(\"commit: %v\", err)}\n\t}\n\n\treturn nil\n}\n\nfunc handleTag(ctx *cli.Context, ctl *client.Client) error {\n\tif ctx.Bool(\"delete\") {\n\t\tname := ctx.Args().Get(0)\n\n\t\tif err := ctl.Untag(name); err != nil {\n\t\t\treturn ExitCode{\n\t\t\t\tUnknownError,\n\t\t\t\tfmt.Sprintf(\"untag: %v\", err),\n\t\t\t}\n\t\t}\n\t} else {\n\t\tif len(ctx.Args()) < 2 {\n\t\t\treturn ExitCode{BadArgs, \"tag needs at least two arguments\"}\n\t\t}\n\n\t\trev := ctx.Args().Get(0)\n\t\tname := ctx.Args().Get(1)\n\n\t\tif err := ctl.Tag(rev, name); err != nil {\n\t\t\treturn ExitCode{\n\t\t\t\tUnknownError,\n\t\t\t\tfmt.Sprintf(\"tag: %v\", err),\n\t\t\t}\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc handleLog(ctx *cli.Context, ctl *client.Client) error {\n\tentries, err := ctl.Log()\n\tif err != nil {\n\t\treturn ExitCode{UnknownError, fmt.Sprintf(\"commit: %v\", err)}\n\t}\n\n\ttmpl, err := readFormatTemplate(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor _, entry := range entries {\n\t\tif tmpl != nil {\n\t\t\tif err := tmpl.Execute(os.Stdout, entry); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tcontinue\n\t\t}\n\n\t\ttags := \"\"\n\t\tisCurr := false\n\t\tif len(entry.Tags) > 0 {\n\t\t\ttags = fmt.Sprintf(\" (%s)\", strings.Join(entry.Tags, \", \"))\n\n\t\t\tfor _, tag := range entry.Tags {\n\t\t\t\tif tag == \"curr\" {\n\t\t\t\t\tisCurr = true\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\tmsg := entry.Msg\n\t\tif msg == \"\" {\n\t\t\tmsg = color.RedString(\"•\")\n\t\t}\n\n\t\tentry.Hash.ShortB58()\n\n\t\tcommitHash := entry.Hash.ShortB58()\n\t\tif isCurr {\n\t\t\tcommitHash = \" - \"\n\t\t}\n\n\t\tfmt.Printf(\n\t\t\t\"%s %s %s%s\\n\",\n\t\t\tcolor.GreenString(commitHash),\n\t\t\tcolor.YellowString(entry.Date.Format(time.UnixDate)),\n\t\t\tmsg,\n\t\t\tcolor.CyanString(tags),\n\t\t)\n\t}\n\n\treturn nil\n}\n" }, { "path": "defaults/defaults.go", "content": "package defaults\n\nimport (\n\t\"os\"\n\n\te \"github.com/pkg/errors\"\n\t\"github.com/sahib/config\"\n)\n\n// CurrentVersion is the current version of brig's config\nconst CurrentVersion = 0\n\n// Defaults is the default validation for brig\nvar Defaults = DefaultsV0\n\n// OpenMigratedConfig takes the config.yml at path and loads it.\n// If required, it also migrates the config structure to the newest\n// version - brig can always rely on the latest config keys to be present.\nfunc OpenMigratedConfig(path string) (*config.Config, error) {\n\tfd, err := os.Open(path) // #nosec\n\tif err != nil {\n\t\treturn nil, e.Wrapf(err, \"failed to open config path %s\", path)\n\t}\n\n\tdefer fd.Close()\n\n\t// NOTE: Add here any migrations with mgr.Add if needed.\n\tmgr := config.NewMigrater(CurrentVersion, config.StrictnessPanic)\n\tmgr.Add(0, nil, DefaultsV0)\n\n\tcfg, err := mgr.Migrate(config.NewYamlDecoder(fd))\n\tif err != nil {\n\t\treturn nil, e.Wrap(err, \"failed to migrate or open\")\n\t}\n\n\treturn cfg, nil\n}\n" }, { "path": "defaults/defaults_v0.go", "content": "package defaults\n\nimport (\n\t\"errors\"\n\t\"net/url\"\n\t\"runtime\"\n\n\t\"github.com/sahib/config\"\n)\n\n// DaemonDefaultURL returns the default URL for the current OS.\nfunc DaemonDefaultURL() string {\n\t// If the platform supports unix sockets,\n\t// we should make use of it.\n\tswitch runtime.GOOS {\n\tcase \"linux\", \"darwin\":\n\t\t// See \"man 7 unix\" - we use an abstract unix domain socket.\n\t\t// This means there is no socket file on the file system.\n\t\t// (other tools use unix:@/path, but Go does not support that notation)\n\t\t// This also means that there are no user rights on the socket file.\n\t\t// If you need this, specify the url in the config.\n\t\treturn \"unix:/tmp/brig.socket?abstract=true\"\n\tdefault:\n\t\treturn \"tcp://127.0.0.1:6666\"\n\t}\n}\n\nfunc urlValidator(val interface{}) error {\n\ts, ok := val.(string)\n\tif !ok {\n\t\treturn errors.New(\"url is not an string\")\n\t}\n\n\t_, err := url.Parse(s)\n\treturn err\n}\n\n// DefaultsV0 is the default config validation for brig\nvar DefaultsV0 = config.DefaultMapping{\n\t\"daemon\": config.DefaultMapping{\n\t\t\"url\": config.DefaultEntry{\n\t\t\tDefault: DaemonDefaultURL(),\n\t\t\tNeedsRestart: true,\n\t\t\tDocs: \"URL of the daemon process.\",\n\t\t\tValidator: urlValidator,\n\t\t},\n\t\t\"ipfs_path_or_url\": config.DefaultEntry{\n\t\t\tDefault: \"\",\n\t\t\tNeedsRestart: true,\n\t\t\tDocs: \"URL or path to the IPFS repository you want to use.\",\n\t\t},\n\t\t\"enable_pprof\": config.DefaultEntry{\n\t\t\tDefault: true,\n\t\t\tNeedsRestart: true,\n\t\t\tDocs: \"Enable a ppropf profile server on startup (see »brig d p --help«)\",\n\t\t},\n\t},\n\t\"events\": config.DefaultMapping{\n\t\t\"enabled\": config.DefaultEntry{\n\t\t\tDefault: true,\n\t\t\tNeedsRestart: false,\n\t\t\tDocs: \"Wether we should handle incoming events and publish auto update events.\",\n\t\t},\n\t\t\"recv_interval\": config.DefaultEntry{\n\t\t\tDefault: \"100ms\",\n\t\t\tNeedsRestart: false,\n\t\t\tDocs: \"Time window in which events are buffered before handling them.\",\n\t\t},\n\t\t\"recv_max_events_per_second\": config.DefaultEntry{\n\t\t\tDefault: 0.5,\n\t\t\tNeedsRestart: false,\n\t\t\tDocs: \"How many incoming events per second to process at max.\",\n\t\t},\n\t\t\"send_interval\": config.DefaultEntry{\n\t\t\tDefault: \"200ms\",\n\t\t\tNeedsRestart: false,\n\t\t\tDocs: \"Time window in which events are buffered before sending them.\",\n\t\t},\n\t\t\"send_max_events_per_second\": config.DefaultEntry{\n\t\t\tDefault: 5.0,\n\t\t\tNeedsRestart: false,\n\t\t\tDocs: \"How many outgoing events per second to send out at max\",\n\t\t},\n\t},\n\t\"gateway\": config.DefaultMapping{\n\t\t\"enabled\": config.DefaultEntry{\n\t\t\tDefault: false,\n\t\t\tNeedsRestart: false,\n\t\t\tDocs: \"Wether the gateway should be running. Will start when enabled.\",\n\t\t},\n\t\t\"port\": config.DefaultEntry{\n\t\t\tDefault: 6001,\n\t\t\tNeedsRestart: false,\n\t\t\tDocs: \"On what port the gateway runs on.\",\n\t\t},\n\t\t\"ui\": config.DefaultMapping{\n\t\t\t\"enabled\": config.DefaultEntry{\n\t\t\t\tDefault: true,\n\t\t\t\tNeedsRestart: false,\n\t\t\t\tDocs: \"Enable the UI. This does not affect the /get endpoint.\",\n\t\t\t},\n\t\t\t\"debug_mode\": config.DefaultEntry{\n\t\t\t\tDefault: false,\n\t\t\t\tNeedsRestart: false,\n\t\t\t\tDocs: \"Enable debug mode (load resources from filesystem).\",\n\t\t\t},\n\t\t},\n\t\t\"auth\": config.DefaultMapping{\n\t\t\t\"anon_allowed\": config.DefaultEntry{\n\t\t\t\tDefault: false,\n\t\t\t\tNeedsRestart: false,\n\t\t\t\tDocs: \"Wether a login is required.\",\n\t\t\t},\n\t\t\t\"anon_user\": config.DefaultEntry{\n\t\t\t\tDefault: \"anon\",\n\t\t\t\tNeedsRestart: false,\n\t\t\t\tDocs: \"What user to copy settings (folder, rights etc.) from.\",\n\t\t\t},\n\t\t\t\"session-encryption-key\": config.DefaultEntry{\n\t\t\t\tDefault: \"\",\n\t\t\t\tNeedsRestart: true,\n\t\t\t\tDocs: \"Encryption key for session cookies. Generated when left empty.\",\n\t\t\t},\n\t\t\t\"session-authentication-key\": config.DefaultEntry{\n\t\t\t\tDefault: \"\",\n\t\t\t\tNeedsRestart: true,\n\t\t\t\tDocs: \"Authentication key for session cookies. Generated when left empty.\",\n\t\t\t},\n\t\t\t\"session-csrf-key\": config.DefaultEntry{\n\t\t\t\tDefault: \"\",\n\t\t\t\tNeedsRestart: true,\n\t\t\t\tDocs: \"Key used for CSRF protection. Generated if empty.\",\n\t\t\t},\n\t\t},\n\t},\n\t\"fs\": config.DefaultMapping{\n\t\t\"sync\": config.DefaultMapping{\n\t\t\t\"ignore_removed\": config.DefaultEntry{\n\t\t\t\tDefault: false,\n\t\t\t\tNeedsRestart: false,\n\t\t\t\tDocs: \"Do not remove what the remote removed.\",\n\t\t\t},\n\t\t\t\"ignore_moved\": config.DefaultEntry{\n\t\t\t\tDefault: false,\n\t\t\t\tNeedsRestart: false,\n\t\t\t\tDocs: \"Do not move what the remote moved\",\n\t\t\t},\n\t\t\t\"pin_added\": config.DefaultEntry{\n\t\t\t\tDefault: false,\n\t\t\t\tNeedsRestart: false,\n\t\t\t\tDocs: \"Do not pin files which were added at the remote\",\n\t\t\t},\n\t\t\t\"conflict_strategy\": config.DefaultEntry{\n\t\t\t\tDefault: \"marker\",\n\t\t\t\tNeedsRestart: false,\n\t\t\t\tValidator: config.EnumValidator(\n\t\t\t\t\t\"marker\", \"ignore\", \"embrace\",\n\t\t\t\t),\n\t\t\t\tDocs: `What strategy to apply in case of conflicts:\n\n * marker: Create a conflict file with the remote's version.\n * ignore: Ignore the remote version completely and keep our version.\n * embrace: Take the remote version and replace ours with it.\n`,\n\t\t\t},\n\t\t},\n\t\t\"pre_cache\": config.DefaultMapping{\n\t\t\t\"enabled\": config.DefaultEntry{\n\t\t\t\tDefault: false,\n\t\t\t\tNeedsRestart: false,\n\t\t\t\tDocs: \"pre-cache files up-on pinning.\",\n\t\t\t},\n\t\t},\n\t\t\"pagecache\": config.DefaultMapping{\n\t\t\t\"max_memory\": config.DefaultEntry{\n\t\t\t\tDefault: \"1G\",\n\t\t\t\tNeedsRestart: true,\n\t\t\t\tDocs: \"Consume at max this amount of memory for the pagecache\",\n\t\t\t},\n\t\t\t\"l2compress\": config.DefaultEntry{\n\t\t\t\tDefault: true,\n\t\t\t\tNeedsRestart: true,\n\t\t\t\tDocs: \"Compress swapped pages over max_memory before going to disk\",\n\t\t\t},\n\t\t},\n\t\t\"repin\": config.DefaultMapping{\n\t\t\t\"enabled\": config.DefaultEntry{\n\t\t\t\tDefault: true,\n\t\t\t\tNeedsRestart: false,\n\t\t\t\tDocs: \"Perform repinning to reclaim space (see »brig pin repin --help«)\",\n\t\t\t},\n\t\t\t\"interval\": config.DefaultEntry{\n\t\t\t\tDefault: \"15m\",\n\t\t\t\tNeedsRestart: false,\n\t\t\t\tDocs: \"In what time interval to trigger repinning automatically.\",\n\t\t\t\tValidator: config.DurationValidator(),\n\t\t\t},\n\t\t\t\"quota\": config.DefaultEntry{\n\t\t\t\tDefault: \"5GB\",\n\t\t\t\tNeedsRestart: false,\n\t\t\t\tDocs: `Maximum stored amount of pinned files to have.\n\n If the quota limit is hit, old versions of a file are unpinned first on the\n next repin. Biggest file first.\n`,\n\t\t\t},\n\t\t\t\"min_depth\": config.DefaultEntry{\n\t\t\t\tDefault: 1,\n\t\t\t\tNeedsRestart: false,\n\t\t\t\tDocs: `Keep at least »n« versions of a pinned file, even if this would exceed the quota.`,\n\t\t\t},\n\t\t\t\"max_depth\": config.DefaultEntry{\n\t\t\t\tDefault: 10,\n\t\t\t\tNeedsRestart: false,\n\t\t\t\tDocs: `Keep at max »n« versions of a pinned file and remove it even if it does not exceed quota.`,\n\t\t\t},\n\t\t\t\"pin_unpinned\": config.DefaultEntry{\n\t\t\t\tDefault: false,\n\t\t\t\tNeedsRestart: false,\n\t\t\t\tDocs: `Pin unpinned files:\n\n * 'true' if you want maximum permitted mirroring\n * 'false' if you want to save traffic\n\n If a file version »n« is such that (min_depth <= »n« < max_depth),\n then the repinner will pin such version if pin_unpinned is set to true.\n Otherwise, it will keep the file unpinned, i.e. not cached at the backend.\n`,\n\t\t\t},\n\t\t},\n\t\t\"autocommit\": config.DefaultMapping{\n\t\t\t\"enabled\": config.DefaultEntry{\n\t\t\t\tDefault: true,\n\t\t\t\tNeedsRestart: false,\n\t\t\t\tDocs: \"Wether to make automatic commits in a fixed interval.\",\n\t\t\t},\n\t\t\t\"interval\": config.DefaultEntry{\n\t\t\t\tDefault: \"5m\",\n\t\t\t\tNeedsRestart: false,\n\t\t\t\tDocs: \"In what interval to make automatic commits.\",\n\t\t\t\tValidator: config.DurationValidator(),\n\t\t\t},\n\t\t},\n\t},\n\t\"repo\": config.DefaultMapping{\n\t\t\"current_user\": config.DefaultEntry{\n\t\t\tDefault: \"\",\n\t\t\tNeedsRestart: false,\n\t\t\tDocs: \"The repository owner that is published to the outside.\",\n\t\t},\n\t\t\"autogc\": config.DefaultMapping{\n\t\t\t\"enabled\": config.DefaultEntry{\n\t\t\t\tDefault: true,\n\t\t\t\tNeedsRestart: false,\n\t\t\t\tDocs: \"Wether to make automatic commits in a fixed interval.\",\n\t\t\t},\n\t\t\t\"interval\": config.DefaultEntry{\n\t\t\t\tDefault: \"60m\",\n\t\t\t\tNeedsRestart: false,\n\t\t\t\tDocs: \"In what interval to make automatic commits.\",\n\t\t\t\tValidator: config.DurationValidator(),\n\t\t\t},\n\t\t},\n\t},\n\t\"mounts\": config.DefaultMapping{\n\t\t// This key stands for the fstab name entry:\n\t\t\"__many__\": config.DefaultMapping{\n\t\t\t\"path\": config.DefaultEntry{\n\t\t\t\tDefault: \"\",\n\t\t\t\tNeedsRestart: true,\n\t\t\t\tDocs: \"The place where the mount path can be found.\",\n\t\t\t},\n\t\t\t\"read_only\": config.DefaultEntry{\n\t\t\t\tDefault: false,\n\t\t\t\tNeedsRestart: true,\n\t\t\t\tDocs: \"Wether this mount should be done read-only.\",\n\t\t\t},\n\t\t\t\"offline\": config.DefaultEntry{\n\t\t\t\tDefault: false,\n\t\t\t\tNeedsRestart: true,\n\t\t\t\tDocs: \"Error out on remote files early if set true.\",\n\t\t\t},\n\t\t\t\"root\": config.DefaultEntry{\n\t\t\t\tDefault: \"/\",\n\t\t\t\tNeedsRestart: true,\n\t\t\t\tDocs: \"The virtual root of the mount.\",\n\t\t\t},\n\t\t},\n\t},\n}\n" }, { "path": "docs/.gitignore", "content": "_build/\n" }, { "path": "docs/Makefile", "content": "# Minimal makefile for Sphinx documentation\n#\n\n# You can set these variables from the command line.\nSPHINXOPTS =\nSPHINXBUILD = sphinx-build\nSPHINXPROJ = brig\nSOURCEDIR = .\nBUILDDIR = _build\n\n# Put it first so that \"make\" without argument is like \"make help\".\nhelp:\n\t@$(SPHINXBUILD) -M help \"$(SOURCEDIR)\" \"$(BUILDDIR)\" $(SPHINXOPTS) $(O)\n\n.PHONY: help Makefile\n\n# Catch-all target: route all unknown targets to Sphinx using the new\n# \"make mode\" option. $(O) is meant as a shortcut for $(SPHINXOPTS).\n%: Makefile\n\t@$(SPHINXBUILD) -M $@ \"$(SOURCEDIR)\" \"$(BUILDDIR)\" $(SPHINXOPTS) $(O)" }, { "path": "docs/_static/css/custom.css", "content": "span.strikethrough { text-decoration: line-through; }\n\na {\n color: #1866bc !important;\n}\n\n#navbar a {\n color: #18bc9c !important;\n}\n\n.alert-info {\n background-color: #5cb7f5a1;\n color: #333;\n}\n\n.alert-warning {\n background-color: #f39c1281;\n color: #333;\n}\n\npre {\n color: #4e5858;\n}\n\n.form-control {\n padding: 1px 2px;\n height: 35px;\n margin-top: 5px;\n}\n\n.navbar-brand {\n font-weight: 900;\n}\n\n.caption-text {\n font-weight: 900;\n}\n\n#sidebar > li {\n display: block;\n}\n\n#sidebar {\n padding-left: 10px;\n position: relative;\n}\n\n/* Make the ``brig`` markup in page color */\n.pre {\n color: #1866bc !important;\n color: #18bc9c !important;\n}\n\n/* Maket the selected item in the sidebar a little darker */\na.current {\n background-color: #00000011;\n}\n\ntbody {\n text-align: left !important;\n}\n" }, { "path": "docs/asciinema/1_init.json", "content": "{\n \"version\": 1,\n \"width\": 119,\n \"height\": 29,\n \"duration\": 27.933128,\n \"command\": null,\n \"title\": null,\n \"env\": {\n \"TERM\": \"xterm-256color\",\n \"SHELL\": \"/bin/zsh\"\n },\n \"stdout\": [\n [\n 0.244236,\n \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"\n ],\n [\n 2.8e-05,\n \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"\n ],\n [\n 0.000153,\n \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"\n ],\n [\n 6e-05,\n \"\\u001b[?1h\\u001b=\"\n ],\n [\n 0.000183,\n \"\\u001b[?2004h\"\n ],\n [\n 1.163009,\n \"\\u001b[4mm\\u001b[24m\"\n ],\n [\n 0.168255,\n \"\\b\\u001b[24m\\u001b[1m\\u001b[31mm\\u001b[1m\\u001b[31mk\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.127064,\n \"\\b\\b\\u001b[1m\\u001b[31mm\\u001b[1m\\u001b[31mk\\u001b[1m\\u001b[31md\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.08503,\n \"\\b\\u001b[1m\\u001b[31md\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.077231,\n \"\\b\\b\\b\\b\\u001b[0m\\u001b[32mm\\u001b[0m\\u001b[32mk\\u001b[0m\\u001b[32md\\u001b[0m\\u001b[32mi\\u001b[32mr\\u001b[39m\"\n ],\n [\n 0.091295,\n \" \"\n ],\n [\n 0.118571,\n \"r\"\n ],\n [\n 0.054292,\n \"e\"\n ],\n [\n 0.048684,\n \"p\"\n ],\n [\n 0.142053,\n \"o\"\n ],\n [\n 0.105353,\n \" \"\n ],\n [\n 0.190366,\n \"&\"\n ],\n [\n 0.148164,\n \"&\"\n ],\n [\n 0.098179,\n \" \"\n ],\n [\n 0.108353,\n \"\\u001b[4mc\\u001b[24m\"\n ],\n [\n 0.082543,\n \"\\b\\u001b[24m\\u001b[32mc\\u001b[32md\\u001b[39m\"\n ],\n [\n 0.066756,\n \" \"\n ],\n [\n 0.402253,\n \"r\"\n ],\n [\n 0.058028,\n \"e\"\n ],\n [\n 0.070225,\n \"p\"\n ],\n [\n 0.139698,\n \"o\"\n ],\n [\n 0.175843,\n \"\\u001b[?1l\\u001b>\"\n ],\n [\n 0.007316,\n \"\\u001b[?2004l\\r\\r\\n\"\n ],\n [\n 0.002185,\n \"\\u001b]2;mkdir repo && cd repo\\u0007\\u001b]1;mkdir\\u0007\"\n ],\n [\n 0.004325,\n \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"\n ],\n [\n 0.000382,\n \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs/repo\\u0007\"\n ],\n [\n 0.000119,\n \"\\u001b]1;..rig/docs/repo\\u0007\"\n ],\n [\n 0.000282,\n \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"\n ],\n [\n 0.000451,\n \"\\u001b[?1h\\u001b=\"\n ],\n [\n 0.001252,\n \"\\u001b[?2004h\"\n ],\n [\n 0.533999,\n \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.099738,\n \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.094967,\n \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.100018,\n \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"\n ],\n [\n 0.060099,\n \" \"\n ],\n [\n 0.380891,\n \"i\"\n ],\n [\n 0.221134,\n \"n\"\n ],\n [\n 0.147805,\n \"i\"\n ],\n [\n 0.041873,\n \"t\"\n ],\n [\n 0.076952,\n \" \"\n ],\n [\n 0.085267,\n \"s\"\n ],\n [\n 0.061555,\n \"a\"\n ],\n [\n 0.042689,\n \"h\"\n ],\n [\n 0.150819,\n \"i\"\n ],\n [\n 0.071275,\n \"b\"\n ],\n [\n 0.438737,\n \"@\"\n ],\n [\n 0.118104,\n \"o\"\n ],\n [\n 0.169356,\n \"n\"\n ],\n [\n 0.388506,\n \"\\b \\b\"\n ],\n [\n 0.226276,\n \"w\"\n ],\n [\n 0.136792,\n \"\\b \\b\"\n ],\n [\n 0.140039,\n \"\\b \\b\"\n ],\n [\n 0.100199,\n \"w\"\n ],\n [\n 0.074795,\n \"a\"\n ],\n [\n 0.089204,\n \"l\"\n ],\n [\n 0.130848,\n \"d\"\n ],\n [\n 0.253754,\n \".\"\n ],\n [\n 0.112072,\n \"d\"\n ],\n [\n 0.047147,\n \"e\"\n ],\n [\n 0.221917,\n \"/\"\n ],\n [\n 0.861882,\n \"l\"\n ],\n [\n 0.245981,\n \"a\"\n ],\n [\n 0.150465,\n \"p\"\n ],\n [\n 0.178583,\n \"t\"\n ],\n [\n 0.064085,\n \"o\"\n ],\n [\n 0.149899,\n \"p\"\n ],\n [\n 0.167082,\n \"\\u001b[?1l\\u001b>\"\n ],\n [\n 0.004906,\n \"\\u001b[?2004l\\r\\r\\n\"\n ],\n [\n 0.001659,\n \"\\u001b]2;brig init sahib@wald.de/laptop\\u0007\\u001b]1;brig\\u0007\"\n ],\n [\n 0.122907,\n \"\\u001b[32m18.02.2018/16:03:09 ⚐\\u001b[0m No Daemon running. Starting daemon from binary: /home/sahib/go/bin/brig\\r\\n\"\n ],\n [\n 0.202807,\n \"\\u001b[J\\u001b[2K\\r\\u001b[31m⊠ New passphrase: \\u001b[0m \\b\"\n ],\n [\n 1.232412,\n \"\\u001b[J\\u001b[2K\\r\\u001b[31m⊠ New passphrase: \\u001b[0m\\u0000\"\n ],\n [\n 0.000696,\n \"\\u001b[J\\u001b[2K\\r\\u001b[31m⊠ New passphrase: \\u001b[0m\\u0000\"\n ],\n [\n 0.30993,\n \"\\u001b[J\\u001b[2K\\r\\u001b[31m⊠ New passphrase: \\u001b[0m\\u0000\\u0000\"\n ],\n [\n 0.000123,\n \"\\u001b[J\\u001b[2K\\r\\u001b[31m⊠ New passphrase: \\u001b[0m\\u0000\\u0000\"\n ],\n [\n 0.149752,\n \"\\u001b[J\\u001b[2K\\r\\u001b[31m⊠ New passphrase: \\u001b[0m\\u0000\\u0000\\u0000\"\n ],\n [\n 0.000538,\n \"\\u001b[J\\u001b[2K\\r\\u001b[31m⊠ New passphrase: \\u001b[0m\\u0000\\u0000\\u0000\"\n ],\n [\n 0.116645,\n \"\\u001b[J\\u001b[2K\\r\"\n ],\n [\n 8.2e-05,\n \"\\u001b[31m⊠ New passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.002332,\n \"\\u001b[J\\u001b[2K\\r\\u001b[35m⊟ New passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.21385,\n \"\\u001b[J\\u001b[2K\\r\"\n ],\n [\n 0.000168,\n \"\\u001b[35m⊟ New passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.000549,\n \"\\u001b[J\\u001b[2K\\r\\u001b[32m⚿ New passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.128442,\n \"\\u001b[J\\u001b[2K\\r\\u001b[32m⚿ New passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.00057,\n \"\\u001b[J\\u001b[2K\\r\\u001b[32m⚿ New passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.116187,\n \"\\u001b[J\\u001b[2K\\r\\u001b[32m⚿ New passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.000762,\n \"\\u001b[J\\u001b[2K\\r\\u001b[32m⚿ New passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.192273,\n \"\\u001b[J\\u001b[2K\\r\\u001b[32m⚿ New passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u001b[J\\u001b[2K\\r\\u001b[32m⚿ New passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.364106,\n \"\\u001b[J\\u001b[2K\\r\\u001b[32m⚿ New passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.000363,\n \"\\u001b[J\\u001b[2K\\r\\u001b[32m⚿ New passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\r\\n\"\n ],\n [\n 0.00064,\n \"Well done! Please re-type your password now for safety:\\r\\n\"\n ],\n [\n 0.000402,\n \"\\u001b[J\\u001b[2K\\r\\u001b[31m⊠ Retype passphrase: \\u001b[0m \\b\"\n ],\n [\n 0.380539,\n \"\\u001b[J\\u001b[2K\\r\\u001b[31m⊠ Retype passphrase: \\u001b[0m\\u0000\"\n ],\n [\n 0.000573,\n \"\\u001b[J\\u001b[2K\\r\\u001b[31m⊠ Retype passphrase: \\u001b[0m\\u0000\"\n ],\n [\n 0.29352,\n \"\\u001b[J\\u001b[2K\\r\\u001b[31m⊠ Retype passphrase: \\u001b[0m\\u0000\\u0000\\u001b[J\\u001b[2K\\r\\u001b[31m⊠ Retype passphrase: \\u001b[0m\\u0000\\u0000\"\n ],\n [\n 0.150925,\n \"\\u001b[J\\u001b[2K\\r\\u001b[31m⊠ Retype passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u001b[J\\u001b[2K\\r\\u001b[31m⊠ Retype passphrase: \\u001b[0m\\u0000\\u0000\\u0000\"\n ],\n [\n 0.114661,\n \"\\u001b[J\\u001b[2K\\r\\u001b[31m⊠ Retype passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.0005,\n \"\\u001b[J\\u001b[2K\\r\\u001b[35m⊟ Retype passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.213871,\n \"\\u001b[J\\u001b[2K\\r\\u001b[35m⊟ Retype passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.000542,\n \"\\u001b[J\\u001b[2K\\r\\u001b[32m⚿ Retype passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.129238,\n \"\\u001b[J\\u001b[2K\\r\\u001b[32m⚿ Retype passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.000605,\n \"\\u001b[J\\u001b[2K\\r\\u001b[32m⚿ Retype passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.121982,\n \"\\u001b[J\\u001b[2K\\r\\u001b[32m⚿ Retype passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.001004,\n \"\\u001b[J\\u001b[2K\\r\\u001b[32m⚿ Retype passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.101944,\n \"\\u001b[J\\u001b[2K\\r\\u001b[32m⚿ Retype passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.001069,\n \"\\u001b[J\\u001b[2K\\r\\u001b[32m⚿ Retype passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\"\n ],\n [\n 0.494948,\n \"\\u001b[J\\u001b[2K\\r\\u001b[32m⚿ Retype passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u001b[J\\u001b[2K\\r\\u001b[32m⚿ Retype passphrase: \\u001b[0m\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\r\\n\"\n ],\n [\n 0.00067,\n \"Estimated time needed to crack password: \\u001b[34m12.0 months\\u001b[0m\\r\\n\\u001b[J\\u001b[2K\\r\"\n ],\n [\n 3.187785,\n \"\\r\\n _____ / /\\\\ ___ / /\\\\ \\r\\n / /::\\\\ / /::\\\\ / /\\\\ / /:/_\\r\\n / /:/\\\\:\\\\ / /:/\\\\:\\\\ / /:/ / /:/ /\\\\ \\r\\n / /:/~/::\\\\ / /:/~/:/ /__/::\\\\ / /:/_/::\\\\ \\r\\n /__/:/ /:/\\\\:| /__/:/ /:/___ \\\\__\\\\/\\\\:\\\\__ /__/:/__\\\\/\\\\:\\\\\\r\\n \\\\ \\\\:\\\\/:/~/:/ \\\\ \\\\:\\\\/:::::/ \\\\ \\\\:\\\\/\\\\ \\\\ \\\\:\\\\ /~~/:/\\r\\n \\\\ \\\\::/ /:/ \\\\ \\\\::/~~~~ \\\\__\\\\::/ \\\\ \\\\:\\\\ /:/\\r\\n \\\\ \\\\:\\\\/:/ \\\\ \\\\:\\\\ /__/:/ \\\\ \\\\:\\\\/:/\\r\\n \\\\ \\\\::/ \\\\ \\\\:\\\\ \\\\__\\\\/ \\\\ \\\\::/\\r\\n \\\\__\\\\/ \\\\__\\\\/ \\\\__\\\\/\\r\\n\\r\\n\\r\\n A new file README.md was automatically added.\\r\\n Use 'brig cat README.md' to view it & get started.\\r\\n\\r\\n\\r\\n\"\n ],\n [\n 0.002162,\n \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"\n ],\n [\n 0.000148,\n \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs/repo\\u0007\\u001b]1;..rig/docs/repo\\u0007\"\n ],\n [\n 5.1e-05,\n \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"\n ],\n [\n 0.000107,\n \"\\u001b[?1h\\u001b=\"\n ],\n [\n 0.000278,\n \"\\u001b[?2004h\"\n ],\n [\n 0.916781,\n \"\\u001b[1m\\u001b[31mn\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.16261,\n \"\\b\\u001b[1m\\u001b[31mn\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.078733,\n \"\\b\\b\\u001b[1m\\u001b[31mn\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mo\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.112002,\n \"\\b\\u001b[1m\\u001b[31mo\\u001b[1m\\u001b[31mg\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.06899,\n \" \"\n ],\n [\n 0.171648,\n \"\\b\"\n ],\n [\n 0.248889,\n \"\\b\\b\\b\\b\\u001b[0m\\u001b[39m \\u001b[0m\\u001b[39m \\u001b[0m\\u001b[39m \\u001b[0m\\u001b[39m \\b\\b\\b\\b\"\n ],\n [\n 0.270981,\n \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.130296,\n \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.110821,\n \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.120156,\n \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"\n ],\n [\n 0.064046,\n \" \"\n ],\n [\n 0.535157,\n \"\\u001b[4ml\\u001b[24m\"\n ],\n [\n 0.112487,\n \"\\b\\u001b[24mls\"\n ],\n [\n 0.568157,\n \"\\u001b[?1l\\u001b>\"\n ],\n [\n 0.00422,\n \"\\u001b[?2004l\\r\\r\\n\"\n ],\n [\n 0.00208,\n \"\\u001b]2;brig ls\\u0007\\u001b]1;brig\\u0007\"\n ],\n [\n 0.160871,\n \"SIZE MODTIME PATH PIN \\r\\n886 B Feb 18 16:03:17 \\u001b[37m/README.md\\u001b[0m \\u001b[36m🖈\\u001b[0m \\r\\n\"\n ],\n [\n 0.001331,\n \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"\n ],\n [\n 7.4e-05,\n \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs/repo\\u0007\"\n ],\n [\n 1.2e-05,\n \"\\u001b]1;..rig/docs/repo\\u0007\"\n ],\n [\n 6.9e-05,\n \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"\n ],\n [\n 7.4e-05,\n \"\\u001b[?1h\\u001b=\"\n ],\n [\n 0.000196,\n \"\\u001b[?2004h\"\n ],\n [\n 1.016178,\n \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.105196,\n \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.10579,\n \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.09966,\n \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"\n ],\n [\n 0.129177,\n \" \"\n ],\n [\n 0.179915,\n \"\\u001b[4mc\\u001b[24m\"\n ],\n [\n 0.058107,\n \"\\b\\u001b[24mca\"\n ],\n [\n 0.146678,\n \"t\"\n ],\n [\n 0.126798,\n \" \"\n ],\n [\n 0.176763,\n \"R\"\n ],\n [\n 0.070224,\n \"E\"\n ],\n [\n 0.134468,\n \"A\"\n ],\n [\n 0.131393,\n \"D\"\n ],\n [\n 0.13735,\n \"M\"\n ],\n [\n 0.091651,\n \"E\"\n ],\n [\n 0.188663,\n \".\"\n ],\n [\n 0.214387,\n \"m\"\n ],\n [\n 0.098478,\n \"d\"\n ],\n [\n 0.198813,\n \"\\u001b[?1l\\u001b>\"\n ],\n [\n 0.006238,\n \"\\u001b[?2004l\\r\\r\\n\"\n ],\n [\n 0.002035,\n \"\\u001b]2;brig cat README.md\\u0007\\u001b]1;brig\\u0007\"\n ],\n [\n 0.122483,\n \"Welcome to brig!\\r\\n\\r\\nHere's what you can do next:\\r\\n\\r\\n • Add a few remotes to sync with (See 'brig remote add -h')\\r\\n • Mount your data somewhere convinient (See 'brig mount -h')\\r\\n • Have a relaxing day exploring brig's features.\\r\\n\\r\\nPlease remember that brig is software in it's very early stages,\\r\\nand will currently eat your data with near-certainty.\\r\\n\\r\\nIf you're done with this README, you can easily remove it:\\r\\n\\r\\n $ brig rm README.md\\r\\n\\r\\n\"\n ],\n [\n 0.001463,\n \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"\n ],\n [\n 0.000111,\n \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs/repo\\u0007\\u001b]1;..rig/docs/repo\\u0007\"\n ],\n [\n 3.6e-05,\n \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"\n ],\n [\n 6.9e-05,\n \"\\u001b[?1h\\u001b=\"\n ],\n [\n 0.00018,\n \"\\u001b[?2004h\"\n ],\n [\n 1.523065,\n \"\\u001b[?2004l\\r\\r\\n\"\n ]\n ]\n}" }, { "path": "docs/asciinema/1_init_with_pwm.json", "content": "{\"version\": 2, \"width\": 172, \"height\": 42, \"timestamp\": 1542382930, \"env\": {\"SHELL\": \"/bin/zsh\", \"TERM\": \"xterm-256color\"}, \"title\": \"brig init (with password manager)\"}\n[0.26352, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[0.26403, \"o\", \"\\u001b]2;sahib@werkbank: /tmp\\u0007\\u001b]1;/tmp\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\\u001b[?1h\\u001b=\\u001b[?2004h\"]\n[0.739097, \"o\", \"\\u001b[4mm\\u001b[24m\"]\n[0.92474, \"o\", \"\\b\\u001b[24m\\u001b[1m\\u001b[31mm\\u001b[1m\\u001b[31mk\\u001b[0m\\u001b[39m\"]\n[1.031303, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mm\\u001b[1m\\u001b[31mk\\u001b[1m\\u001b[31md\\u001b[0m\\u001b[39m\"]\n[1.153579, \"o\", \"\\b\\u001b[1m\\u001b[31md\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[1.230782, \"o\", \"\\b\\b\\b\\b\\u001b[0m\\u001b[32mm\\u001b[0m\\u001b[32mk\\u001b[0m\\u001b[32md\\u001b[0m\\u001b[32mi\\u001b[32mr\\u001b[39m\"]\n[1.310904, \"o\", \" \"]\n[1.579276, \"o\", \"\\u001b[4mr\\u001b[24m\"]\n[1.642426, \"o\", \"\\b\\u001b[24mre\"]\n[1.712467, \"o\", \"p\"]\n[1.860411, \"o\", \"o\"]\n[1.993894, \"o\", \" \"]\n[3.034364, \"o\", \"&\"]\n[3.155707, \"o\", \"&\"]\n[3.263644, \"o\", \" \"]\n[3.395157, \"o\", \"\\u001b[1m\\u001b[31mc\\u001b[0m\\u001b[39m\"]\n[3.462327, \"o\", \"\\b\\u001b[0m\\u001b[32mc\\u001b[32md\\u001b[39m\"]\n[3.539217, \"o\", \" \"]\n[3.645307, \"o\", \"\\u001b[4mr\\u001b[24m\"]\n[3.707485, \"o\", \"\\b\\u001b[24mre\"]\n[3.772131, \"o\", \"p\"]\n[3.95697, \"o\", \"o\"]\n[4.725546, \"o\", \"\\u001b[?1l\\u001b>\"]\n[4.734682, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[4.737525, \"o\", \"\\u001b]2;mkdir repo && cd repo\\u0007\\u001b]1;mkdir\\u0007\"]\n[4.742294, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[4.742882, \"o\", \"\\u001b]2;sahib@werkbank: /tmp/repo\\u0007\"]\n[4.743388, \"o\", \"\\u001b]1;/tmp/repo\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[4.744018, \"o\", \"\\u001b[?1h\\u001b=\"]\n[4.745864, \"o\", \"\\u001b[?2004h\"]\n[5.92003, \"o\", \"\\u001b[1m\\u001b[31mp\\u001b[0m\\u001b[39m\"]\n[6.056376, \"o\", \"\\b\\u001b[1m\\u001b[31mp\\u001b[1m\\u001b[31ma\\u001b[0m\\u001b[39m\"]\n[6.124477, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mp\\u001b[1m\\u001b[31ma\\u001b[1m\\u001b[31ms\\u001b[0m\\u001b[39m\"]\n[6.290954, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mp\\u001b[0m\\u001b[32ma\\u001b[0m\\u001b[32ms\\u001b[32ms\\u001b[39m\"]\n[6.399263, \"o\", \" \"]\n[7.291905, \"o\", \"g\"]\n[7.360923, \"o\", \"e\"]\n[7.422143, \"o\", \"n\"]\n[7.499221, \"o\", \"e\"]\n[7.544024, \"o\", \"r\"]\n[7.653468, \"o\", \"a\"]\n[7.775172, \"o\", \"t\"]\n[7.826824, \"o\", \"e\"]\n[7.905244, \"o\", \" \"]\n[8.187297, \"o\", \"b\"]\n[8.307228, \"o\", \"r\"]\n[8.406541, \"o\", \"i\"]\n[8.503768, \"o\", \"g\"]\n[8.74212, \"o\", \"/\"]\n[8.926867, \"o\", \"r\"]\n[9.007904, \"o\", \"e\"]\n[9.047081, \"o\", \"p\"]\n[9.200125, \"o\", \"o\"]\n[9.764218, \"o\", \"/\"]\n[9.933345, \"o\", \"s\"]\n[9.984954, \"o\", \"a\"]\n[10.128667, \"o\", \"h\"]\n[10.30151, \"o\", \"i\"]\n[10.396692, \"o\", \"b\"]\n[10.772247, \"o\", \"@\"]\n[11.629835, \"o\", \"w\"]\n[11.71424, \"o\", \"a\"]\n[11.820773, \"o\", \"l\"]\n[11.951813, \"o\", \"d\"]\n[12.115615, \"o\", \".\"]\n[12.210083, \"o\", \"d\"]\n[12.266417, \"o\", \"e\"]\n[12.876332, \"o\", \" \"]\n[13.199804, \"o\", \"\\b\"]\n[14.158845, \"o\", \"/\"]\n[14.369763, \"o\", \";\"]\n[14.48393, \"o\", \"\\u001b[1m\\u001b[31ma\\u001b[0m\\u001b[39m\"]\n[14.799511, \"o\", \"\\b\\u001b[0m\\u001b[39m \\b\"]\n[14.928714, \"o\", \"\\b \\b\"]\n[15.09851, \"o\", \"l\"]\n[15.216493, \"o\", \"a\"]\n[15.303913, \"o\", \"p\"]\n[15.434103, \"o\", \"t\"]\n[15.513938, \"o\", \"o\"]\n[15.672982, \"o\", \"p\"]\n[15.795216, \"o\", \" \"]\n[15.997295, \"o\", \"-\"]\n[16.210899, \"o\", \"n\"]\n[16.289418, \"o\", \" \"]\n[16.457636, \"o\", \"2\"]\n[16.59324, \"o\", \"0\"]\n[17.222134, \"o\", \"\\u001b[?1l\\u001b>\"]\n[17.227172, \"o\", \"\\u001b[?2004l\"]\n[17.227446, \"o\", \"\\r\\r\\n\"]\n[17.229032, \"o\", \"\\u001b]2;pass generate brig/repo/sahib@wald.de/laptop -n 20\\u0007\"]\n[17.22927, \"o\", \"\\u001b]1;pass\\u0007\"]\n[17.25874, \"o\", \"mkdir: Verzeichnis '/home/sahib/.password-store/brig/repo/sahib@wald.de' angelegt\\r\\n\"]\n[17.273215, \"o\", \"\\u001b[1mThe generated password for \\u001b[4mbrig/repo/sahib@wald.de/laptop\\u001b[24m is:\\u001b[0m\\r\\n\\u001b[1m\\u001b[93mwRQUbpoYCCwZIJxkn7VV\\u001b[0m\\r\\n\"]\n[17.273828, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[17.273942, \"o\", \"\\u001b]2;sahib@werkbank: /tmp/repo\\u0007\"]\n[17.273967, \"o\", \"\\u001b]1;/tmp/repo\\u0007\"]\n[17.274064, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[17.274153, \"o\", \"\\u001b[?1h\\u001b=\"]\n[17.274413, \"o\", \"\\u001b[?2004h\"]\n[18.856296, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[18.940748, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[19.061012, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[19.159053, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[19.233445, \"o\", \" \"]\n[21.190964, \"o\", \"i\"]\n[21.386602, \"o\", \"n\"]\n[21.538857, \"o\", \"i\"]\n[21.609502, \"o\", \"t\"]\n[21.683962, \"o\", \" \"]\n[22.683931, \"o\", \" \"]\n[22.997219, \"o\", \"\\b\"]\n[23.738281, \"o\", \"s\"]\n[23.799633, \"o\", \"a\"]\n[23.859625, \"o\", \"h\"]\n[24.023621, \"o\", \"i\"]\n[24.106901, \"o\", \"b\"]\n[24.445686, \"o\", \"@\"]\n[24.668167, \"o\", \"w\"]\n[24.747857, \"o\", \"a\"]\n[24.836444, \"o\", \"l\"]\n[24.964705, \"o\", \"d\"]\n[25.107596, \"o\", \".\"]\n[25.213753, \"o\", \"d\"]\n[25.304183, \"o\", \"e\"]\n[25.855981, \"o\", \"/\"]\n[26.067438, \"o\", \"l\"]\n[26.19953, \"o\", \"a\"]\n[26.324293, \"o\", \"p\"]\n[26.537115, \"o\", \"t\"]\n[26.626544, \"o\", \"o\"]\n[26.82602, \"o\", \"p\"]\n[26.984607, \"o\", \" \"]\n[27.63086, \"o\", \"-\"]\n[27.910446, \"o\", \"w\"]\n[28.066763, \"o\", \" \"]\n[28.24338, \"o\", \"\\u001b[33m'\\u001b[39m\"]\n[28.476716, \"o\", \"\\b\\u001b[33m'\\u001b[33mp\\u001b[39m\"]\n[28.548911, \"o\", \"\\b\\u001b[33mp\\u001b[33ma\\u001b[39m\"]\n[28.591751, \"o\", \"\\b\\u001b[33ma\\u001b[33ms\\u001b[39m\"]\n[28.74535, \"o\", \"\\b\\u001b[33ms\\u001b[33ms\\u001b[39m\"]\n[28.821941, \"o\", \"\\b\\u001b[33ms\\u001b[33m \\u001b[39m\"]\n[31.700107, \"o\", \"\\u001b[40D\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[1m\\u001b[31mg\\u001b[1m\\u001b[31m \\u001b[1m\\u001b[31mi\\u001b[1m\\u001b[31mn\\u001b[1m\\u001b[31mi\\u001b[1m\\u001b[31mt\\u001b[1m\\u001b[31m \\u001b[1m\\u001b[31ms\\u001b[1m\\u001b[31ma\\u001b[1m\\u001b[31mh\\u001b[1m\\u001b[31mi\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31m@\\u001b[1m\\u001b[31mw\\u001b[1m\\u001b[31ma\\u001b[1m\\u001b[31ml\\u001b[1m\\u001b[31md\\u001b[1m\\u001b[31m.\\u001b[1m\\u001b[31md\\u001b[1m\\u001b[31me\\u001b[1m\\u001b[31m/\\u001b[1m\\u001b[31ml\\u001b[1m\\u001b[31ma\\u001b[1m\\u001b[31mp\\u001b[1m\\u001b[31mt\\u001b[1m\\u001b[31mo\\u001b[1m\\u001b[31mp\\u001b[1m\\u001b[31m \\u001b[1m\\u001b[31m-\\u001b[1m\\u001b[31mw\\u001b[1m\\u001b[31m \\u001b[1m\\u001b[31m'\\u001b[1m\\u001b[31mp\\u001b[1m\\u001b[31ma\\u001b[1m\\u001b[31ms\\u001b[1m\\u001b[31ms\\u001b[1m\\u001b[31m \\u001b[1m\\u001b[31mbrig/repo/sahib@wald.de/laptop\\u001b[0m\\u001b[27m\\u001b[39m\"]\n[32.642035, \"o\", \"\\u001b[70D\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[0m\\u001b[32mg\\u001b[39m\\u001b[0m\\u001b[39m \\u001b[0m\\u001b[39mi\\u001b[0m\\u001b[39mn\\u001b[0m\\u001b[39mi\\u001b[0m\\u001b[39mt\\u001b[0m\\u001b[39m \\u001b[0m\\u001b[39ms\\u001b[0m\\u001b[39ma\\u001b[0m\\u001b[39mh\\u001b[0m\\u001b[39mi\\u001b[0m\\u001b[39mb\\u001b[0m\\u001b[39m@\\u001b[0m\\u001b[39mw\\u001b[0m\\u001b[39ma\\u001b[0m\\u001b[39ml\\u001b[0m\\u001b[39md\\u001b[0m\\u001b[39m.\\u001b[0m\\u001b[39md\\u001b[0m\\u001b[39me\\u001b[0m\\u001b[39m/\\u001b[0m\\u001b[39ml\\u001b[0m\\u001b[39ma\\u001b[0m\\u001b[39mp\\u001b[0m\\u001b[39mt\\u001b[0m\\u001b[39mo\\u001b[0m\\u001b[39mp\\u001b[0m\\u001b[39m \\u001b[0m\\u001b[39m-\\u001b[0m\\u001b[39mw\\u001b[0m\\u001b[39m \\u001b[0m\\u001b[33m'\\u001b[0m\\u001b[33mp\\u001b[0m\\u001b[33ma\\u001b[0m\\u001b[33ms\\u001b[0m\\u001b[33ms\\u001b[0m\\u001b[33m \\u001b[0m\\u001b[33mb\\u001b[0m\\u001b[33mr\\u001b[0m\\u001b[33mi\\u001b[0m\\u001b[33mg\\u001b[0m\\u001b[33m/\\u001b[0m\\u001b[33mr\\u001b[0m\\u001b[33me\\u001b[0m\\u001b[33mp\\u001b[0m\\u001b[33mo\\u001b[0m\\u001b[33m/\\u001b[0m\\u001b[33ms\\u001b[0m\\u001b[33ma\\u001b[0m\\u001b[33mh\\u001b[0m\\u001b[33mi\\u001b[0m\\u001b[33mb\\u001b[0m\\u001b[33m@\\u001b[0m\\u001b[33mw\\u001b[0m\\u001b[33ma\\u001b[0m\\u001b[33ml\\u001b[0m\\u001b[33md\\u001b[0m\\u001b[33m.\\u001b[0m\\u001b[33md\\u001b[0m\\u001b[33me\\u001b[0m\\u001b[33m/\\u001b[0m\\u001b[33ml\\u001b[0m\\u001b[33ma\\u001b[0m\\u001b[33mp\\u001b[0m\\u001b[33mt\\u001b[0m\\u001b[33mo\\u001b[0m\\u001b[33mp\\u001b[33m'\\u001b[39m\"]\n[33.456599, \"o\", \"\\u001b[?1l\\u001b>\"]\n[33.458004, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[33.458685, \"o\", \"\\u001b]2;brig init sahib@wald.de/laptop -w 'pass brig/repo/sahib@wald.de/laptop'\\u0007\\u001b]1;brig\\u0007\"]\n[33.843016, \"o\", \"Guessed folder for init: /tmp/repo\\r\\n\"]\n[36.537341, \"o\", \"\\r\\n _____ / /\\\\ ___ / /\\\\ \\r\\n / /::\\\\ / /::\\\\ / /\\\\ / /:/_\\r\\n / /:/\\\\:\\\\ / /:/\\\\:\\\\ / /:/ / /:/ /\\\\ \\r\\n / /:/~/::\\\\ / /:/~/:/ /__/::\\\\ / /:/_/::\\\\ \\r\\n /__/:/ /:/\\\\:| /__/:/ /:/___ \\\\__\\\\/\\\\:\\\\__ /__/:/__\\\\/\\\\:\\\\\\r\\n \\\\ \\\\:\\\\/:/~/:/ \\\\ \\\\:\\\\/:::::/ \\\\ \\\\:\\\\/\\\\ \\\\ \\\\:\\\\ /~~/:/\\r\\n \\\\ \\\\::/ /:/ \\\\ \\\\::/~~~~ \\\\__\\\\::/ \\\\ \\\\:\\\\ /:/\\r\\n \\\\ \\\\:\\\\/:/ \\\\ \\\\:\\\\ /__/:/ \\\\ \\\\:\\\\/:/\\r\\n \\\\ \\\\::/ \\\\ \\\\:\\\\ \\\\__\\\\/ \\\\ \\\\::/\\r\\n \\\\__\\\\/ \\\\__\\\\/ \\\\__\\\\/\\r\\n\\r\\n\\r\\n A new file README.md was automatically added.\\r\\n Use 'brig cat README.md' to view it & get started.\\r\\n\\r\\n\\r\\n\"]\n[36.543616, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[36.543888, \"o\", \"\\u001b]2;sahib@werkbank: /tmp/repo\\u0007\\u001b]1;/tmp/repo\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\\u001b[?1h\\u001b=\"]\n[36.544159, \"o\", \"\\u001b[?2004h\"]\n[39.488223, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n" }, { "path": "docs/asciinema/2_adding.json", "content": "{\n \"version\": 1,\n \"width\": 119,\n \"height\": 29,\n \"duration\": 21.32594,\n \"command\": null,\n \"title\": null,\n \"env\": {\n \"TERM\": \"xterm-256color\",\n \"SHELL\": \"/bin/zsh\"\n },\n \"stdout\": [\n [\n 0.25386,\n \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"\n ],\n [\n 2.7e-05,\n \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs/repo\\u0007\\u001b]1;..rig/docs/repo\\u0007\"\n ],\n [\n 0.000118,\n \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"\n ],\n [\n 9.8e-05,\n \"\\u001b[?1h\\u001b=\"\n ],\n [\n 0.000186,\n \"\\u001b[?2004h\"\n ],\n [\n 0.453974,\n \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.105389,\n \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.085149,\n \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.092634,\n \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"\n ],\n [\n 0.053191,\n \" \"\n ],\n [\n 0.120726,\n \"s\"\n ],\n [\n 0.138693,\n \"t\"\n ],\n [\n 0.104973,\n \"a\"\n ],\n [\n 0.113068,\n \"g\"\n ],\n [\n 0.033543,\n \"e\"\n ],\n [\n 0.086594,\n \" \"\n ],\n [\n 0.108291,\n \"\\u001b[4mm\\u001b[24m\"\n ],\n [\n 0.162977,\n \"\\b\\u001b[4mm\\u001b[4mu\\u001b[24m\"\n ],\n [\n 0.112732,\n \"\\b\\u001b[4mu\\u001b[4ms\\u001b[24m\"\n ],\n [\n 0.234469,\n \"\\b\\u001b[4ms\\u001b[4mic.mp3\\u001b[24m \"\n ],\n [\n 0.725932,\n \"\\u001b[?1l\\u001b>\"\n ],\n [\n 0.005389,\n \"\\u001b[?2004l\\r\\r\\n\"\n ],\n [\n 0.002298,\n \"\\u001b]2;brig stage music.mp3\\u0007\\u001b]1;brig\\u0007\"\n ],\n [\n 0.462899,\n \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"\n ],\n [\n 4.1e-05,\n \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs/repo\\u0007\"\n ],\n [\n 1.1e-05,\n \"\\u001b]1;..rig/docs/repo\\u0007\"\n ],\n [\n 6.7e-05,\n \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"\n ],\n [\n 0.0001,\n \"\\u001b[?1h\\u001b=\"\n ],\n [\n 0.000226,\n \"\\u001b[?2004h\"\n ],\n [\n 0.688362,\n \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.115037,\n \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.088531,\n \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.101543,\n \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"\n ],\n [\n 0.066626,\n \" \"\n ],\n [\n 0.109212,\n \"\\u001b[4ml\\u001b[24m\"\n ],\n [\n 0.125075,\n \"\\b\\u001b[24mls\"\n ],\n [\n 0.105769,\n \"\\u001b[?1l\\u001b>\"\n ],\n [\n 0.006356,\n \"\\u001b[?2004l\\r\\r\\n\"\n ],\n [\n 0.001889,\n \"\\u001b]2;brig ls\\u0007\\u001b]1;brig\\u0007\"\n ],\n [\n 0.181364,\n \"SIZE MODTIME \"\n ],\n [\n 0.00013,\n \"PATH PIN \\r\\n886 B Feb 18 16:03:17 \\u001b[37m/README.md\\u001b[0m \\u001b[36m🖈\\u001b[0m \\r\\n\\u001b[33m13 MB\\u001b[0m Feb 18 16:05:45 \\u001b[37m/music.mp3\\u001b[0m \\u001b[36m🖈\\u001b[0m \\r\\n\"\n ],\n [\n 0.001523,\n \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"\n ],\n [\n 0.000107,\n \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs/repo\\u0007\"\n ],\n [\n 1.2e-05,\n \"\\u001b]1;..rig/docs/repo\\u0007\"\n ],\n [\n 8.5e-05,\n \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"\n ],\n [\n 0.000104,\n \"\\u001b[?1h\\u001b=\"\n ],\n [\n 0.000267,\n \"\\u001b[?2004h\"\n ],\n [\n 0.618057,\n \"\\u001b[32mbrig\\u001b[39m ls\"\n ],\n [\n 0.228909,\n \"\\b\\b \\b\\b\"\n ],\n [\n 0.192473,\n \"t\"\n ],\n [\n 0.144214,\n \"r\"\n ],\n [\n 0.038708,\n \"e\"\n ],\n [\n 0.165568,\n \"e\"\n ],\n [\n 0.182023,\n \"\\u001b[?1l\\u001b>\"\n ],\n [\n 0.004345,\n \"\\u001b[?2004l\\r\\r\\n\"\n ],\n [\n 0.001945,\n \"\\u001b]2;brig tree\\u0007\\u001b]1;brig\\u0007\"\n ],\n [\n 0.11428,\n \"\\u001b[35m•\\u001b[0m \\u001b[36m🖈\\u001b[0m\\r\\n├── music.mp3 \\u001b[36m🖈\\u001b[0m\\r\\n└── README.md \\u001b[36m🖈\\u001b[0m\\r\\n\\r\\n1 directory, 2 files\\r\\n\"\n ],\n [\n 0.001242,\n \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"\n ],\n [\n 8.5e-05,\n \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs/repo\\u0007\"\n ],\n [\n 1.3e-05,\n \"\\u001b]1;..rig/docs/repo\\u0007\"\n ],\n [\n 7.3e-05,\n \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"\n ],\n [\n 8.3e-05,\n \"\\u001b[?1h\\u001b=\"\n ],\n [\n 0.000189,\n \"\\u001b[?2004h\"\n ],\n [\n 1.986702,\n \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.139791,\n \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.061285,\n \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.1216,\n \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"\n ],\n [\n 0.059354,\n \" \"\n ],\n [\n 1.141574,\n \"\\u001b[4mc\\u001b[24m\"\n ],\n [\n 0.049557,\n \"\\b\\u001b[24mca\"\n ],\n [\n 0.159666,\n \"t\"\n ],\n [\n 0.068815,\n \" \"\n ],\n [\n 0.146648,\n \"\\u001b[4mm\\u001b[24m\"\n ],\n [\n 0.155988,\n \"\\b\\u001b[4mm\\u001b[4mu\\u001b[24m\"\n ],\n [\n 0.079945,\n \"\\b\\u001b[4mu\\u001b[4ms\\u001b[24m\"\n ],\n [\n 0.091685,\n \"\\b\\u001b[4ms\\u001b[4mi\\u001b[24m\"\n ],\n [\n 0.075616,\n \"\\b\\u001b[4mi\\u001b[4mc\\u001b[24m\"\n ],\n [\n 0.139947,\n \"\\b\\u001b[4mc\\u001b[4m.\\u001b[24m\"\n ],\n [\n 0.180603,\n \"\\b\\u001b[4m.\\u001b[4mm\\u001b[24m\"\n ],\n [\n 0.163774,\n \"\\b\\u001b[4mm\\u001b[4mp\\u001b[24m\"\n ],\n [\n 0.06169,\n \"\\b\\u001b[4mp\\u001b[4m3\\u001b[24m\"\n ],\n [\n 0.646438,\n \" \"\n ],\n [\n 0.192937,\n \"|\"\n ],\n [\n 0.128012,\n \" \"\n ],\n [\n 0.355076,\n \"\\u001b[4mm\\u001b[24m\"\n ],\n [\n 0.185119,\n \"\\b\\u001b[24m\\u001b[1m\\u001b[31mm\\u001b[1m\\u001b[31mp\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.079648,\n \"\\b\\b\\u001b[0m\\u001b[32mm\\u001b[0m\\u001b[32mp\\u001b[32mv\\u001b[39m\"\n ],\n [\n 0.069737,\n \" \"\n ],\n [\n 0.098894,\n \"-\"\n ],\n [\n 0.442838,\n \"\\u001b[?1l\\u001b>\\u001b[?2004l\\r\\r\\n\\u001b]2;brig cat music.mp3 | mpv -\\u0007\\u001b]1;brig\\u0007\"\n ],\n [\n 0.119644,\n \"\\u001b[0mPlaying: -\\r\\n\\u001b[0m\"\n ],\n [\n 0.006226,\n \"\\u001b[0m[file] Reading from stdin...\\r\\n\\u001b[0m\"\n ],\n [\n 0.019648,\n \"\\u001b[0;33m[ffmpeg/demuxer] mp3: invalid concatenated file detected - using bitrate for duration\\r\\n\\u001b[0m\"\n ],\n [\n 0.00472,\n \"\\u001b[0m (+) Audio --aid=1 (mp3 2ch 44100Hz)\\r\\n\"\n ],\n [\n 8.2e-05,\n \"\\u001b[0m\\u001b[0mFile tags:\\r\\n\"\n ],\n [\n 2.1e-05,\n \"\\u001b[0m\\u001b[0m Artist: Epica\\r\\n\\u001b[0m\"\n ],\n [\n 1.7e-05,\n \"\\u001b[0m Album: The Classical Conspiracy: Live in Miskolc, Hungary\\r\\n\\u001b[0m\\u001b[0m Album_Artist: Epica\\r\\n\"\n ],\n [\n 1.8e-05,\n \"\\u001b[0m\\u001b[0m Title: The Imperial March\\r\\n\\u001b[0m\\u001b[0m Track: 8/18\\r\\n\"\n ],\n [\n 1.3e-05,\n \"\\u001b[0m\"\n ],\n [\n 0.004403,\n \"\\u001b[0mAO: [pulse] 44100Hz stereo 2ch s16\\r\\n\\u001b[0m\"\n ],\n [\n 0.015081,\n \"\\r\\u001b[K\\u001b[0mA: 00:00:00 / 00:00:00 Cache: 10s+5MB\\r\\u001b[0m\"\n ],\n [\n 0.675774,\n \"\\r\\u001b[K\\u001b[0mA: 00:00:00 / 00:00:00 Cache: 9s+5MB\\r\\u001b[0m\"\n ],\n [\n 0.203763,\n \"\\r\\u001b[K\\u001b[0mA: 00:00:00 / 00:00:00 Cache: 10s+5MB\\r\\u001b[0m\"\n ],\n [\n 0.203299,\n \"\\r\\u001b[K\\u001b[0mA: 00:00:01 / 00:00:00 Cache: 10s+5MB\\r\\u001b[0m\"\n ],\n [\n 0.385072,\n \"\\r\\u001b[K\\u001b[0mA: 00:00:01 / 00:00:00 Cache: 9s+5MB\\r\\u001b[0m\"\n ],\n [\n 0.126352,\n \"\\r\\u001b[K\\u001b[0mA: 00:00:01 / 00:00:00 Cache: 10s+5MB\\r\\u001b[0m\"\n ],\n [\n 0.404842,\n \"\\r\\u001b[K\\u001b[0mA: 00:00:02 / 00:00:00 Cache: 10s+5MB\\r\\u001b[0m\"\n ],\n [\n 0.202959,\n \"\\r\\u001b[K\\u001b[0mA: 00:00:02 / 00:00:00 (1%) Cache: 10s+5MB\\r\\u001b[0m\"\n ],\n [\n 0.813401,\n \"\\r\\u001b[K\\u001b[0mA: 00:00:03 / 00:00:00 (1%) Cache: 10s+5MB\\r\\u001b[0m\"\n ],\n [\n 0.407844,\n \"\\r\\u001b[K\\u001b[0mA: 00:00:03 / 00:00:00 (1%) Cache: 9s+5MB\\r\\u001b[0m\"\n ],\n [\n 0.079101,\n \"\\r\\u001b[K\\u001b[0mA: 00:00:03 / 00:00:00 (1%) Cache: 10s+5MB\\r\\u001b[0m\"\n ],\n [\n 0.126073,\n \"\\r\\u001b[K\\u001b[0mA: 00:00:03 / 00:00:00 (2%) Cache: 10s+5MB\\r\\u001b[0m\"\n ],\n [\n 0.40592,\n \"\\r\\u001b[K\\u001b[0mA: 00:00:04 / 00:00:00 (2%) Cache: 10s+5MB\\r\\u001b[0m\"\n ],\n [\n 1.015693,\n \"\\r\\u001b[K\\u001b[0mA: 00:00:05 / 00:00:00 (2%) Cache: 10s+5MB\\r\\u001b[0m\"\n ],\n [\n 0.609984,\n \"\\r\\u001b[K\\u001b[0mA: 00:00:05 / 00:00:00 (3%) Cache: 9s+5MB\\r\\u001b[0m\"\n ],\n [\n 0.079399,\n \"\\r\\u001b[K\\u001b[0mA: 00:00:05 / 00:00:00 (3%) Cache: 10s+5MB\\r\\u001b[0m\"\n ],\n [\n 0.254169,\n \"^C\"\n ],\n [\n 0.006464,\n \"\\r\\n\\u001b[0m\\r\\n\\u001b[0m\"\n ],\n [\n 0.000298,\n \"\\u001b[0m\\r\\n\\u001b[0m\\u001b[0mExiting... (Quit)\\r\\n\\u001b[0m\"\n ],\n [\n 0.022616,\n \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"\n ],\n [\n 0.000604,\n \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs/repo\\u0007\"\n ],\n [\n 0.000122,\n \"\\u001b]1;..rig/docs/repo\\u0007\"\n ],\n [\n 0.001088,\n \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\\u001b[?1h\\u001b=\"\n ],\n [\n 0.001661,\n \"\\u001b[?2004h\"\n ],\n [\n 1.256398,\n \"\\u001b[?2004l\\r\\r\\n\"\n ]\n ]\n}" }, { "path": "docs/asciinema/3_coreutils.json", "content": "{\n \"version\": 1,\n \"width\": 119,\n \"height\": 29,\n \"duration\": 44.439839,\n \"command\": null,\n \"title\": null,\n \"env\": {\n \"TERM\": \"xterm-256color\",\n \"SHELL\": \"/bin/zsh\"\n },\n \"stdout\": [\n [\n 0.207775,\n \"b\"\n ],\n [\n 0.03976,\n \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"\n ],\n [\n 0.000112,\n \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"\n ],\n [\n 0.000121,\n \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"\n ],\n [\n 6.6e-05,\n \"\\u001b[?1h\\u001b=\"\n ],\n [\n 8.2e-05,\n \"\\u001b[?2004h\"\n ],\n [\n 0.001595,\n \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.104775,\n \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.103078,\n \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.171794,\n \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"\n ],\n [\n 0.084843,\n \" \"\n ],\n [\n 0.111081,\n \"\\u001b[4mm\\u001b[24m\"\n ],\n [\n 0.2013,\n \"\\b\\u001b[24mmk\"\n ],\n [\n 0.213333,\n \"d\"\n ],\n [\n 0.116987,\n \"i\"\n ],\n [\n 0.096428,\n \"r\"\n ],\n [\n 0.108839,\n \" \"\n ],\n [\n 0.615141,\n \"s\"\n ],\n [\n 0.112813,\n \"u\"\n ],\n [\n 0.093527,\n \"b\"\n ],\n [\n 0.458499,\n \"\\u001b[?1l\\u001b>\"\n ],\n [\n 0.006789,\n \"\\u001b[?2004l\\r\\r\\n\"\n ],\n [\n 0.0028,\n \"\\u001b]2;brig mkdir sub\\u0007\\u001b]1;brig\\u0007\"\n ],\n [\n 0.112551,\n \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"\n ],\n [\n 6e-05,\n \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\"\n ],\n [\n 1.4e-05,\n \"\\u001b]1;~/dev/brig/docs\\u0007\"\n ],\n [\n 5.9e-05,\n \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"\n ],\n [\n 7.3e-05,\n \"\\u001b[?1h\\u001b=\"\n ],\n [\n 0.000193,\n \"\\u001b[?2004h\"\n ],\n [\n 1.144128,\n \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.151721,\n \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.099278,\n \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.113866,\n \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"\n ],\n [\n 0.071336,\n \" \"\n ],\n [\n 0.12183,\n \"\\u001b[4mc\\u001b[24m\"\n ],\n [\n 0.085112,\n \"\\b\\u001b[24mcp\"\n ],\n [\n 0.076426,\n \" \"\n ],\n [\n 0.118376,\n \"\\u001b[4mm\\u001b[24m\"\n ],\n [\n 0.150092,\n \"\\b\\u001b[24mmu\"\n ],\n [\n 0.061787,\n \"s\"\n ],\n [\n 0.111346,\n \"i\"\n ],\n [\n 0.102251,\n \"c\"\n ],\n [\n 0.134833,\n \".\"\n ],\n [\n 0.208191,\n \"m\"\n ],\n [\n 0.234634,\n \"p\"\n ],\n [\n 0.096577,\n \"3\"\n ],\n [\n 0.163669,\n \" \"\n ],\n [\n 0.315213,\n \"s\"\n ],\n [\n 0.110484,\n \"u\"\n ],\n [\n 0.097823,\n \"b\"\n ],\n [\n 0.185982,\n \" \"\n ],\n [\n 0.3964,\n \"\\b\"\n ],\n [\n 0.377085,\n \"/\"\n ],\n [\n 0.556364,\n \"\\b \\b\"\n ],\n [\n 0.255609,\n \"\\u001b[?1l\\u001b>\"\n ],\n [\n 0.007065,\n \"\\u001b[?2004l\\r\\r\\n\"\n ],\n [\n 0.002288,\n \"\\u001b]2;brig cp music.mp3 sub\\u0007\\u001b]1;brig\\u0007\"\n ],\n [\n 0.116393,\n \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"\n ],\n [\n 6.1e-05,\n \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\"\n ],\n [\n 1.4e-05,\n \"\\u001b]1;~/dev/brig/docs\\u0007\"\n ],\n [\n 6e-05,\n \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"\n ],\n [\n 6.4e-05,\n \"\\u001b[?1h\\u001b=\"\n ],\n [\n 0.000187,\n \"\\u001b[?2004h\"\n ],\n [\n 1.240271,\n \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.1047,\n \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.102508,\n \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.088436,\n \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"\n ],\n [\n 0.086373,\n \" \"\n ],\n [\n 0.138829,\n \"t\"\n ],\n [\n 0.166321,\n \"r\"\n ],\n [\n 0.074024,\n \"e\"\n ],\n [\n 0.177903,\n \"e\"\n ],\n [\n 0.169713,\n \"\\u001b[?1l\\u001b>\"\n ],\n [\n 0.003955,\n \"\\u001b[?2004l\\r\\r\\n\"\n ],\n [\n 0.001864,\n \"\\u001b]2;brig tree\\u0007\\u001b]1;brig\\u0007\"\n ],\n [\n 0.123081,\n \"\\u001b[35m•\\u001b[0m \\u001b[36m🖈\\u001b[0m\\r\\n├── README.md \\u001b[36m🖈\\u001b[0m\\r\\n\"\n ],\n [\n 9e-05,\n \"└──\\u001b[32msub\\u001b[0m \\u001b[36m🖈\\u001b[0m\\r\\n └── music.mp3 \\u001b[36m🖈\\u001b[0m\\r\\n\\r\\n2 directories, 3 files\\r\\n\"\n ],\n [\n 0.001329,\n \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"\n ],\n [\n 7.5e-05,\n \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\"\n ],\n [\n 1.2e-05,\n \"\\u001b]1;~/dev/brig/docs\\u0007\"\n ],\n [\n 6.1e-05,\n \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \"\n ],\n [\n 1.1e-05,\n \"\\u001b[K\"\n ],\n [\n 7.3e-05,\n \"\\u001b[?1h\\u001b=\"\n ],\n [\n 0.00019,\n \"\\u001b[?2004h\"\n ],\n [\n 2.752774,\n \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.218615,\n \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.15312,\n \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.47546,\n \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"\n ],\n [\n 0.091406,\n \" \"\n ],\n [\n 0.851159,\n \"\\u001b[4mi\\u001b[24m\"\n ],\n [\n 0.153946,\n \"\\b\\u001b[4mi\\u001b[4mn\\u001b[24m\"\n ],\n [\n 0.113271,\n \"\\b\\b\\u001b[24mi\\u001b[24mnf\"\n ],\n [\n 0.070447,\n \"o\"\n ],\n [\n 0.064971,\n \" \"\n ],\n [\n 0.216765,\n \"R\"\n ],\n [\n 0.069948,\n \"E\"\n ],\n [\n 0.089236,\n \"A\"\n ],\n [\n 0.136061,\n \"D\"\n ],\n [\n 0.12703,\n \"M\"\n ],\n [\n 0.08556,\n \"E\"\n ],\n [\n 0.179354,\n \".\"\n ],\n [\n 0.223757,\n \"m\"\n ],\n [\n 0.09138,\n \"d\"\n ],\n [\n 0.121202,\n \"\\u001b[?1l\\u001b>\"\n ],\n [\n 0.00509,\n \"\\u001b[?2004l\\r\\r\\n\"\n ],\n [\n 0.001736,\n \"\\u001b]2;brig info README.md\\u0007\\u001b]1;brig\\u0007\"\n ],\n [\n 0.117613,\n \"ATTR VALUE\"\n ],\n [\n 3.4e-05,\n \" \\r\\n\\u001b[37mPath\\u001b[0m /README.md \\r\\n\\u001b[37mType\\u001b[0m file \\r\\n\\u001b[37mSize\\u001b[0m 886 B \\r\\n\\u001b[37mHash\\u001b[0m SEfXUBbsLgBV1KQtogr9N3WxLuNWYACVoyZbqqDvrCYWgRusPSMTBcfXgveekZh6qNBEYxaFPXZq6mncayPg5yc55z8rp \\r\\n\\u001b[37mInode\\u001b[0m \"\n ],\n [\n 6.5e-05,\n \"4 \"\n ],\n [\n 4.7e-05,\n \" \\r\\n\\u001b[37mPinned\\u001b[0m \\u001b[32myes\\u001b[0m \\r\\n\\u001b[37mModTime\\u001b[0m 2018-02-18T16:03:17+01:00 \\r\\n\\u001b[37mContent\\u001b[0m QmRD1CzUEBg8BMTArBJymVHs8MVZAYAHi2tzA1UxXqf2qp \"\n ],\n [\n 4.5e-05,\n \" \\r\\n\"\n ],\n [\n 0.001377,\n \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"\n ],\n [\n 7.7e-05,\n \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\"\n ],\n [\n 1.2e-05,\n \"\\u001b]1;~/dev/brig/docs\\u0007\"\n ],\n [\n 5.9e-05,\n \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"\n ],\n [\n 8.6e-05,\n \"\\u001b[?1h\\u001b=\"\n ],\n [\n 0.000226,\n \"\\u001b[?2004h\"\n ],\n [\n 2.408108,\n \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.112957,\n \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.110455,\n \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.097714,\n \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"\n ],\n [\n 0.243174,\n \"\\b\\b\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[1m\\u001b[31mg\\u001b[1m\\u001b[31me\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.341565,\n \"\\b\\b\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[0m\\u001b[32mg\\u001b[39m\\u001b[0m\\u001b[39m \\b\"\n ],\n [\n 0.084114,\n \" \"\n ],\n [\n 0.060533,\n \"\\u001b[4me\\u001b[24m\"\n ],\n [\n 0.103896,\n \"\\b\\u001b[24med\"\n ],\n [\n 0.049514,\n \"i\"\n ],\n [\n 0.172611,\n \"t\"\n ],\n [\n 0.090499,\n \" \"\n ],\n [\n 0.237876,\n \"R\"\n ],\n [\n 0.073149,\n \"E\"\n ],\n [\n 0.131375,\n \"A\"\n ],\n [\n 0.111012,\n \"D\"\n ],\n [\n 0.146386,\n \"M\"\n ],\n [\n 0.088129,\n \"E\"\n ],\n [\n 0.165009,\n \".\"\n ],\n [\n 0.216731,\n \"m\"\n ],\n [\n 0.083528,\n \"d\"\n ],\n [\n 1.476078,\n \"\\b\\b\"\n ],\n [\n 0.264974,\n \"\\b\\b\\b\\b\\b\\b\\b\"\n ],\n [\n 0.575857,\n \"\\b\"\n ],\n [\n 0.294863,\n \"\\b\\u001b[P\\u001b[10C \\u001b[11D\"\n ],\n [\n 0.198429,\n \"\\b\\u001b[P\\u001b[10C \\u001b[11D\"\n ],\n [\n 0.169154,\n \"t README.md\\u001b[10D\"\n ],\n [\n 0.118885,\n \"i README.md\\u001b[10D\"\n ],\n [\n 0.214733,\n \"\\u001b[?1l\\u001b>\"\n ],\n [\n 0.005064,\n \"\\u001b[?2004l\\r\\r\\n\"\n ],\n [\n 0.001899,\n \"\\u001b]2;brig edti README.md\\u0007\\u001b]1;brig\\u0007\"\n ],\n [\n 0.103904,\n \"`\\u001b[31medti\\u001b[0m` is not a valid command. Did you maybe mean `\\u001b[32medit\\u001b[0m`?\\r\\n\"\n ],\n [\n 0.001328,\n \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"\n ],\n [\n 7.2e-05,\n \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\"\n ],\n [\n 1.3e-05,\n \"\\u001b]1;~/dev/brig/docs\\u0007\"\n ],\n [\n 5.9e-05,\n \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"\n ],\n [\n 7e-05,\n \"\\u001b[?1h\\u001b=\"\n ],\n [\n 0.000196,\n \"\\u001b[?2004h\"\n ],\n [\n 0.869697,\n \"\\u001b[32mbrig\\u001b[39m edti README.md\"\n ],\n [\n 0.269395,\n \"\\u001b[19D\"\n ],\n [\n 0.134928,\n \"\\u001b[5C\"\n ],\n [\n 0.176458,\n \"\\u001b[5C\"\n ],\n [\n 0.380294,\n \"\\b\"\n ],\n [\n 0.37218,\n \"\\b\\u001b[P\\u001b[10C \\u001b[11D\"\n ],\n [\n 0.147017,\n \"\\b\\u001b[P\\u001b[10C \\u001b[11D\"\n ],\n [\n 0.163162,\n \"i README.md\\u001b[10D\"\n ],\n [\n 0.081396,\n \"t README.md\\u001b[10D\"\n ],\n [\n 0.73976,\n \"\\u001b[?1l\\u001b>\"\n ],\n [\n 0.005398,\n \"\\u001b[?2004l\\r\\r\\n\"\n ],\n [\n 0.001748,\n \"\\u001b]2;brig edit README.md\\u0007\\u001b]1;brig\\u0007\"\n ],\n [\n 0.127872,\n \"\\u001b[?1000h\\u001b[?2004h\\u001b[?1049h\\u001b[22;0;0t\\u001b[?1h\\u001b=\\u001b[?2004h\"\n ],\n [\n 0.000239,\n \"\\u001b[1;29r\\u001b[?12h\\u001b[?12l\\u001b[27m\\u001b[23m\\u001b[29m\\u001b[m\\u001b[H\\u001b[2J\\u001b[?25l\\u001b[29;1H\\\"/tmp/brig-cmd-buffer-1263340949723681570.md\\\"\"\n ],\n [\n 5.3e-05,\n \" 15L, 443C\"\n ],\n [\n 0.012203,\n \"\\u001b[2;1H▽\\u001b[6n\\u001b[2;1H \\u001b[1;1H\\u001b[>c\\u001b]10;?\\u0007\\u001b]11;?\\u0007\"\n ],\n [\n 0.002133,\n \"\\u001b[1;1HWelcome to brig!\\r\\n\\r\\nHere's what you can do next:\\u001b[5;5H•\\u001b[5;7HAdd a few remotes to sync with (See 'brig remote add -h')\\r\\n •\\u001b[6;7HMount your data somewhere convinient (See 'brig mount -h')\\r\\n •\\u001b[7;7HHave a relaxing day exploring brig's features.\\r\\n\\r\\nPlease remember that brig is software in it's very early stages,\\r\\nand will currently eat your data with near-certainty.\\r\\n\\r\\nIf you're done with this README, you can easily remove it:\\u001b[14;5H$ brig rm README.md\\r\\n\\r\\n\\u001b[94m~ \\u001b[17;1H~ \\u001b[18;1H~ \\u001b[19;1H~ \\u001b[20;1H~ \"\n ],\n [\n 3e-05,\n \" \\u001b[21;1H~ \\u001b[22;1H~ \\u001b[23;1H~ \\u001b[24;1H~ \\u001b[25;1H~ \\u001b[26;1H~ \\u001b[27;1H~ \\u001b[28;1H~ \"\n ],\n [\n 9e-06,\n \" \\u001b[m\\u001b[29;102H1,1\\u001b[9CAlles\\u001b[1;1H\\u001b[?25h\"\n ],\n [\n 0.011277,\n \"\\u001bP+q436f\\u001b\\\\\\u001bP+q6b75\\u001b\\\\\\u001bP+q6b64\\u001b\\\\\\u001bP+q6b72\\u001b\\\\\\u001bP+q6b6c\\u001b\\\\\\u001bP+q2332\\u001b\\\\\\u001bP+q2334\\u001b\\\\\\u001bP+q2569\\u001b\\\\\\u001bP+q2a37\\u001b\\\\\\u001bP+q6b31\\u001b\\\\\"\n ],\n [\n 0.002856,\n \"\\u001b[?1000l\\u001b[?1006h\\u001b[?1002h\\u001b[?1006l\\u001b[?1002l\\u001b[?1006h\\u001b[?1002h\\u001b[27m\\u001b[23m\\u001b[29m\\u001b[m\\u001b[H\\u001b[2J\\u001b[?25l\\u001b[1;1HWelcome to brig!\\r\\n\\r\\nHere's what you can do next:\\u001b[5;5H•\\u001b[5;7HAdd a few remotes to sync with (See 'brig remote add -h')\\r\\n •\\u001b[6;7HMount your data somewhere convinient (See 'brig mount -h')\\r\\n •\\u001b[7;7HHave a relaxing day exploring brig's features.\\r\\n\\r\\nPlease remember that brig is software in it's very early stages,\\r\\nand will currently eat your data with near-certainty.\\r\\n\\r\\nIf you're done with this README, you can easily remove it:\\u001b[14;5H$ brig rm README.md\\r\\n\\r\\n\\u001b[94m~ \\u001b[17;1H~ \\u001b[18;1H~ \\u001b[19;1H~ \"\n ],\n [\n 5.3e-05,\n \" \\u001b[20;1H~ \\u001b[21;1H~ \\u001b[22;1H~ \\u001b[23;1H~ \\u001b[24;1H~ \\u001b[25;1H~ \\u001b[26;1H~ \\u001b[27;1H~ \"\n ],\n [\n 2.9e-05,\n \" \\u001b[28;1H~ \\u001b[m\\u001b[29;102H1,1\\u001b[9CAlles\\r\\\"/tmp/brig-cmd-buffer-1263340949723681570.md\\\" 15L, 443C\\u001b[1;1H\\u001b[?25h\"\n ],\n [\n 0.61525,\n \"\\u001b[?25l\\u001b[29;92Hj\\u001b[1;1H\"\n ],\n [\n 0.000361,\n \"\\u001b[29;92H \\u001b[2;1H\\u001b[29;102H2,0-1\\u001b[2;1H\\u001b[?25h\"\n ],\n [\n 0.307898,\n \"\\u001b[?25l\\u001b[29;92HG\\u001b[2;1H\"\n ],\n [\n 0.000155,\n \"\\u001b[29;92H \\u001b[15;1H\\u001b[29;102H15,0-1\\u001b[15;1H\\u001b[?25h\"\n ],\n [\n 0.548891,\n \"\\u001b[?25l\\u001b[29;92Ho\\u001b[15;1H\"\n ],\n [\n 0.000101,\n \"\\u001b[29;92H \\u001b[16;1H\"\n ],\n [\n 0.000179,\n \"\\u001b[29;1H\\u001b[1m-- EINFÜGEN --\\u001b[m\\u001b[29;15H\\u001b[K\\u001b[29;102H16,1\\u001b[8CAlles\"\n ],\n [\n 0.001464,\n \"\\u001b[16;1H\\u001b[K\\u001b[16;1H\\u001b[?25h\"\n ],\n [\n 0.195215,\n \"\\u001b[?25l\\u001b[17;1H\\u001b[K\\u001b[29;103H7\\u001b[17;1H\\u001b[?25h\"\n ],\n [\n 0.411704,\n \"\\u001b[?25lI\\u001b[29;105H2\\u001b[17;2H\\u001b[?25h\"\n ],\n [\n 0.234594,\n \"\\u001b[?25l'\\u001b[29;105H3\\u001b[17;3H\\u001b[?25h\"\n ],\n [\n 0.122519,\n \"\\u001b[?25lv\\u001b[29;105H4\\u001b[17;4H\\u001b[?25h\"\n ],\n [\n 0.066522,\n \"\\u001b[?25le\\u001b[29;105H5\\u001b[17;5H\\u001b[?25h\"\n ],\n [\n 0.112979,\n \"\\u001b[?25l\\u001b[29;105H6\\u001b[17;6H\\u001b[?25h\"\n ],\n [\n 0.312782,\n \"\\u001b[?25le\\u001b[29;105H7\\u001b[17;7H\\u001b[?25h\"\n ],\n [\n 0.154464,\n \"\\u001b[?25ld\\u001b[29;105H8\\u001b[17;8H\\u001b[?25h\"\n ],\n [\n 0.168844,\n \"\\u001b[?25li\\u001b[29;105H9\\u001b[17;9H\\u001b[?25h\"\n ],\n [\n 0.090198,\n \"\\u001b[?25lt\\u001b[29;105H10\\u001b[17;10H\\u001b[?25h\"\n ],\n [\n 0.089237,\n \"\\u001b[?25le\\u001b[29;106H1\\u001b[17;11H\\u001b[?25h\"\n ],\n [\n 0.129701,\n \"\\u001b[?25lr\\u001b[29;106H2\\u001b[17;12H\\u001b[?25h\"\n ],\n [\n 0.439249,\n \"\\u001b[?25l\\u001b[17;11H\\u001b[K\\u001b[29;106H1\\u001b[17;11H\\u001b[?25h\"\n ],\n [\n 0.080156,\n \"\\u001b[?25ld\\u001b[29;106H2\\u001b[17;12H\\u001b[?25h\"\n ],\n [\n 0.091909,\n \"\\u001b[?25l\\u001b[29;106H3\\u001b[17;13H\\u001b[?25h\"\n ],\n [\n 0.168699,\n \"\\u001b[?25ly\\u001b[29;106H4\\u001b[17;14H\\u001b[?25h\"\n ],\n [\n 0.104238,\n \"\\u001b[?25le\\u001b[29;106H5\\u001b[17;15H\\u001b[?25h\"\n ],\n [\n 0.220856,\n \"\\u001b[?25l\\u001b[17;14H\\u001b[K\\u001b[29;106H4\\u001b[17;14H\\u001b[?25h\"\n ],\n [\n 0.14326,\n \"\\u001b[?25l\\u001b[17;13H\\u001b[K\\u001b[29;106H3\\u001b[17;13H\\u001b[?25h\"\n ],\n [\n 0.142547,\n \"\\u001b[?25li\\u001b[29;106H4\\u001b[17;14H\\u001b[?25h\"\n ],\n [\n 0.089015,\n \"\\u001b[?25lt\\u001b[29;106H5\\u001b[17;15H\\u001b[?25h\"\n ],\n [\n 0.101241,\n \"\\u001b[?25l.\\u001b[29;106H6\\u001b[17;16H\\u001b[?25h\"\n ],\n [\n 0.148249,\n \"\\u001b[29;1H\\u001b[K\\u001b[17;15H\\u001b[?25l\\u001b[29;92H^[\\u001b[17;15H\"\n ],\n [\n 0.100582,\n \"\\u001b[29;92H \\u001b[17;16H\"\n ],\n [\n 0.000733,\n \"\\u001b[29;102H17,15\\u001b[7CAlles\\u001b[17;15H\\u001b[?25h\"\n ],\n [\n 0.18371,\n \"\\u001b[?25l\\u001b[29;92H:\\u001b[17;15H\\u001b[29;92H\\u001b[K\\u001b[29;1H:\\u001b[?2004h\\u001b[?25h\"\n ],\n [\n 0.067401,\n \"w\\u001b[?25l\\u001b[?25h\"\n ],\n [\n 0.068066,\n \"q\"\n ],\n [\n 1.9e-05,\n \"\\u001b[?25l\\u001b[?25h\"\n ],\n [\n 0.061677,\n \"\\r\"\n ],\n [\n 4.3e-05,\n \"\\u001b[?25l\\u001b[?1006l\\u001b[?1002l\\u001b[?2004l\"\n ],\n [\n 3.8e-05,\n \"\\\"/tmp/brig-cmd-buffer-1263340949723681570.md\\\"\"\n ],\n [\n 8e-05,\n \" 17L, 460C geschrieben\"\n ],\n [\n 0.000507,\n \"\\r\\r\\r\\n\\u001b[?2004l\\u001b[?1l\\u001b>\\u001b[?25h\\u001b[?1049l\\u001b[23;0;0t\"\n ],\n [\n 0.135995,\n \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"\n ],\n [\n 0.000115,\n \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\"\n ],\n [\n 1.9e-05,\n \"\\u001b]1;~/dev/brig/docs\\u0007\"\n ],\n [\n 0.000112,\n \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"\n ],\n [\n 0.000149,\n \"\\u001b[?1h\\u001b=\"\n ],\n [\n 0.000335,\n \"\\u001b[?2004h\"\n ],\n [\n 1.145903,\n \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.128894,\n \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.085092,\n \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"\n ],\n [\n 0.104495,\n \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"\n ],\n [\n 0.07748,\n \" \"\n ],\n [\n 0.150049,\n \"\\u001b[4mc\\u001b[24m\"\n ],\n [\n 0.065485,\n \"\\b\\u001b[24mca\"\n ],\n [\n 0.151361,\n \"t\"\n ],\n [\n 0.07143,\n \" \"\n ],\n [\n 0.798179,\n \"R\"\n ],\n [\n 0.049034,\n \"E\"\n ],\n [\n 0.148152,\n \"A\"\n ],\n [\n 0.101871,\n \"D\"\n ],\n [\n 0.147676,\n \"M\"\n ],\n [\n 0.08569,\n \"E\"\n ],\n [\n 0.501451,\n \".\"\n ],\n [\n 0.181451,\n \"m\"\n ],\n [\n 0.090745,\n \"d\"\n ],\n [\n 0.115721,\n \"\\u001b[?1l\\u001b>\"\n ],\n [\n 0.005991,\n \"\\u001b[?2004l\\r\\r\\n\"\n ],\n [\n 0.001998,\n \"\\u001b]2;brig cat README.md\\u0007\\u001b]1;brig\\u0007\"\n ],\n [\n 0.113793,\n \"Welcome to brig!\\r\\n\\r\\nHere's what you can do next:\\r\\n\\r\\n • Add a few remotes to sync with (See 'brig remote add -h')\\r\\n • Mount your data somewhere convinient (See 'brig mount -h')\\r\\n • Have a relaxing day exploring brig's features.\\r\\n\\r\\nPlease remember that brig is software in it's very early stages,\\r\\nand will currently eat your data with near-certainty.\\r\\n\\r\\nIf you're done with this README, you can easily remove it:\\r\\n\\r\\n $ brig rm README.md\\r\\n\\r\\n\\r\\nI've edited it.\\r\\n\"\n ],\n [\n 0.00158,\n \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"\n ],\n [\n 8.5e-05,\n \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\"\n ],\n [\n 1.4e-05,\n \"\\u001b]1;~/dev/brig/docs\\u0007\"\n ],\n [\n 7.7e-05,\n \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"\n ],\n [\n 7.7e-05,\n \"\\u001b[?1h\\u001b=\"\n ],\n [\n 0.000193,\n \"\\u001b[?2004h\"\n ],\n [\n 1.198824,\n \"\\u001b[32mbrig\\u001b[39m cat README.md\"\n ],\n [\n 0.183351,\n \"\\u001b[13Dedit README.md\"\n ],\n [\n 0.414961,\n \"\\u001b[12Dti\\u001b[10C\"\n ],\n [\n 0.409659,\n \"\\u001b[14Dinfo\\u001b[10C\"\n ],\n [\n 0.534112,\n \"\\u001b[?1l\\u001b>\"\n ],\n [\n 0.005484,\n \"\\u001b[?2004l\\r\\r\\n\"\n ],\n [\n 0.001893,\n \"\\u001b]2;brig info README.md\\u0007\\u001b]1;brig\\u0007\"\n ],\n [\n 0.117828,\n \"ATTR VALUE \"\n ],\n [\n 2.7e-05,\n \" \\r\\n\\u001b[37mPath\\u001b[0m /README.md \\r\\n\\u001b[37mType\\u001b[0m file \\r\\n\\u001b[37mSize\\u001b[0m 460 B \\r\\n\\u001b[37mHash\\u001b[0m SEfXUEBxWHqPU8J2cn4mg2bBoB48Wrpba2DUwsyuGaDc2nYfgBLZ6S3Bs4iY4kiCUJfa2KWSNx8gEBh2FMbh2D1yCNxst \\r\\n\\u001b[37mInode\\u001b[0m 4 \\r\\n\\u001b[37mPinned\\u001b[0m\"\n ],\n [\n 1.3e-05,\n \" \\u001b[32myes\\u001b[0m \"\n ],\n [\n 1e-05,\n \" \\r\\n\\u001b[37mModTime\\u001b[0m 2018-02-18T21:51:40+01:00 \"\n ],\n [\n 9e-06,\n \" \\r\\n\"\n ],\n [\n 1e-05,\n \"\\u001b[37mContent\\u001b[0m QmQMGt4QGSGhBXnNAHD3pYymmyxjV7dS7nax3F4wCf2Uut \"\n ],\n [\n 1e-05,\n \" \\r\\n\"\n ],\n [\n 0.001378,\n \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"\n ],\n [\n 6.7e-05,\n \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\"\n ],\n [\n 1.6e-05,\n \"\\u001b]1;~/dev/brig/docs\\u0007\"\n ],\n [\n 6.4e-05,\n \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"\n ],\n [\n 7.7e-05,\n \"\\u001b[?1h\\u001b=\"\n ],\n [\n 0.000187,\n \"\\u001b[?2004h\"\n ],\n [\n 1.715623,\n \"\\u001b[?2004l\\r\\r\\n\"\n ]\n ]\n}" }, { "path": "docs/asciinema/4_mount.json", "content": "{\"version\": 2, \"width\": 119, \"height\": 29, \"timestamp\": 1519811299, \"env\": {\"SHELL\": \"/bin/zsh\", \"TERM\": \"xterm-256color\"}}\n[0.259676, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[0.259761, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[0.259893, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[0.260008, \"o\", \"\\u001b[?1h\\u001b=\"]\n[0.260247, \"o\", \"\\u001b[?2004h\"]\n[0.682887, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[0.801996, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[0.891365, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[0.986864, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[1.037332, \"o\", \" \"]\n[1.149093, \"o\", \"\\u001b[4ml\\u001b[24m\"]\n[1.267448, \"o\", \"\\b\\u001b[24mls\"]\n[1.513916, \"o\", \"\\u001b[?1l\\u001b>\"]\n[1.518176, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[1.520369, \"o\", \"\\u001b]2;brig ls\\u0007\\u001b]1;brig\\u0007\"]\n[1.631909, \"o\", \"SIZE MODTIME \"]\n[1.632051, \"o\", \" PATH PIN \\r\\n886 B Feb 28 10:45:21 \\u001b[37m/README.md\\u001b[0m \\u001b[36m🖈\\u001b[0m \\r\\n\\u001b[33m13 MB\\u001b[0m Feb 28 10:46:48 \\u001b[32m/sub\\u001b[0m \\u001b[36m🖈\\u001b[0m \\r\\n\"]\n[1.633428, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[1.63352, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\"]\n[1.633546, \"o\", \"\\u001b]1;~/dev/brig/docs\\u0007\"]\n[1.633623, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[1.633823, \"o\", \"\\u001b[?1h\\u001b=\"]\n[1.633919, \"o\", \"\\u001b[?2004h\"]\n[2.603338, \"o\", \"\\u001b[4mm\\u001b[24m\"]\n[2.796509, \"o\", \"\\b\\u001b[24m\\u001b[1m\\u001b[31mm\\u001b[1m\\u001b[31mk\\u001b[0m\\u001b[39m\"]\n[2.907083, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mm\\u001b[1m\\u001b[31mk\\u001b[1m\\u001b[31md\\u001b[0m\\u001b[39m\"]\n[2.996988, \"o\", \"\\b\\u001b[1m\\u001b[31md\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[3.074828, \"o\", \"\\b\\b\\b\\b\\u001b[0m\\u001b[32mm\\u001b[0m\\u001b[32mk\\u001b[0m\\u001b[32md\\u001b[0m\\u001b[32mi\\u001b[32mr\\u001b[39m\"]\n[3.15677, \"o\", \" \"]\n[3.247834, \"o\", \"\\u001b[4m/\\u001b[24m\"]\n[3.646365, \"o\", \"\\b\\u001b[4m/\\u001b[4mt\\u001b[24m\"]\n[3.74645, \"o\", \"\\b\\u001b[4mt\\u001b[4mm\\u001b[24m\"]\n[3.948814, \"o\", \"\\b\\u001b[4mm\\u001b[4mp\\u001b[24m\"]\n[4.289082, \"o\", \"\\b\\u001b[4mp\\u001b[4m/\\u001b[24m\"]\n[4.576632, \"o\", \"\\b\\b\\b\\b\\b\\u001b[24m/\\u001b[24mt\\u001b[24mm\\u001b[24mp\\u001b[24m/m\"]\n[4.75591, \"o\", \"o\"]\n[4.931808, \"o\", \"u\"]\n[5.100663, \"o\", \"n\"]\n[5.192704, \"o\", \"t\"]\n[5.461691, \"o\", \"\\u001b[?1l\\u001b>\"]\n[5.466679, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[5.468092, \"o\", \"\\u001b]2;mkdir /tmp/mount\\u0007\"]\n[5.468757, \"o\", \"\\u001b]1;mkdir\\u0007\"]\n[5.474787, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[5.475931, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[5.476488, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\\u001b[?1h\\u001b=\"]\n[5.477887, \"o\", \"\\u001b[?2004h\"]\n[8.033329, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[8.134549, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[8.228743, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[8.321856, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[8.365702, \"o\", \" \"]\n[8.637156, \"o\", \"\\u001b[4mm\\u001b[24m\"]\n[8.942413, \"o\", \"\\b\\u001b[24mmo\"]\n[9.131874, \"o\", \"u\"]\n[9.297628, \"o\", \"n\"]\n[9.389473, \"o\", \"t\"]\n[9.460171, \"o\", \" \"]\n[9.532387, \"o\", \"\\u001b[4m/\\u001b[24m\"]\n[9.647129, \"o\", \"\\b\\u001b[4m/\\u001b[4mt\\u001b[24m\"]\n[9.742548, \"o\", \"\\b\\u001b[4mt\\u001b[4mm\\u001b[24m\"]\n[10.265361, \"o\", \"\\b\\u001b[4mm\\u001b[4mp\\u001b[24m\"]\n[10.523571, \"o\", \"\\b\\u001b[4mp\\u001b[4m/\\u001b[24m\"]\n[10.764126, \"o\", \"\\b\\u001b[4m/\\u001b[4mm\\u001b[24m\"]\n[10.940644, \"o\", \"\\b\\u001b[4mm\\u001b[4mo\\u001b[24m\"]\n[11.11384, \"o\", \"\\b\\u001b[4mo\\u001b[4mu\\u001b[24m\"]\n[11.274619, \"o\", \"\\b\\u001b[4mu\\u001b[4mn\\u001b[24m\"]\n[11.336741, \"o\", \"\\b\\u001b[4mn\\u001b[4mt\\u001b[24m\"]\n[11.501596, \"o\", \"\\u001b[?1l\\u001b>\"]\n[11.5099, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[11.511935, \"o\", \"\\u001b]2;brig mount /tmp/mount\\u0007\\u001b]1;brig\\u0007\"]\n[11.630285, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[11.63042, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\"]\n[11.630489, \"o\", \"\\u001b]1;~/dev/brig/docs\\u0007\"]\n[11.630544, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[11.630595, \"o\", \"\\u001b[?1h\\u001b=\"]\n[11.630847, \"o\", \"\\u001b[?2004h\"]\n[11.783534, \"o\", \"\\u001b[4mc\\u001b[24m\"]\n[11.846931, \"o\", \"\\b\\u001b[24m\\u001b[32mc\\u001b[32md\\u001b[39m\"]\n[11.874495, \"o\", \" \"]\n[12.0069, \"o\", \"\\u001b[4m/\\u001b[24m\"]\n[12.139978, \"o\", \"\\b\\u001b[4m/\\u001b[4mt\\u001b[24m\"]\n[12.262404, \"o\", \"\\b\\u001b[4mt\\u001b[4mm\\u001b[24m\"]\n[12.368633, \"o\", \"\\b\\u001b[4mm\\u001b[4mp\\u001b[1m\\u001b[4m/\\u001b[0m\\u001b[24m\"]\n[12.526367, \"o\", \"\\b\\u001b[0m\\u001b[4m/\\u001b[4mm\\u001b[24m\"]\n[12.718498, \"o\", \"\\b\\u001b[4mm\\u001b[4mo\\u001b[24m\"]\n[12.890811, \"o\", \"\\b\\u001b[4mo\\u001b[4mu\\u001b[24m\"]\n[12.979988, \"o\", \"\\b\\u001b[4mu\\u001b[4mnt\\u001b[1m\\u001b[4m/\\u001b[0m\\u001b[24m\"]\n[13.185149, \"o\", \"\\b\\b\\u001b[4mt\\u001b[24m\\u001b[0m\\u001b[24m \\b\"]\n[13.186246, \"o\", \"\\u001b[?1l\\u001b>\"]\n[13.191054, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[13.192883, \"o\", \"\\u001b]2;cd /tmp/mount\\u0007\\u001b]1;cd\\u0007\"]\n[13.193619, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[13.193984, \"o\", \"\\u001b]2;sahib@werkbank: /tmp/mount\\u0007\\u001b]1;/tmp/mount\\u0007\"]\n[13.194845, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\\u001b[?1h\\u001b=\"]\n[13.195642, \"o\", \"\\u001b[?2004h\"]\n[13.358323, \"o\", \"\\u001b[32ml\\u001b[39m\"]\n[13.500286, \"o\", \"\\b\\u001b[32ml\\u001b[32ms\\u001b[39m\"]\n[13.623484, \"o\", \"\\u001b[?1l\\u001b>\"]\n[13.626994, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[13.628768, \"o\", \"\\u001b]2;ls --color=tty\\u0007\\u001b]1;ls\\u0007\"]\n[13.640118, \"o\", \"\\u001b[0m\\u001b[35mREADME.md\\u001b[0m \\u001b[1msub\\u001b[0m\\r\\n\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\\u001b]2;sahib@werkbank: /tmp/mount\\u0007\\u001b]1;/tmp/mount\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \"]\n[13.640771, \"o\", \"\\u001b[K\"]\n[13.641637, \"o\", \"\\u001b[?1h\\u001b=\"]\n[13.642956, \"o\", \"\\u001b[?2004h\"]\n[15.590359, \"o\", \" \"]\n[15.871315, \"o\", \"\\b\"]\n[16.642904, \"o\", \"\\u001b[1m\\u001b[31mv\\u001b[0m\\u001b[39m\"]\n[16.831186, \"o\", \"\\b\\u001b[0m\\u001b[32mv\\u001b[32mi\\u001b[39m\"]\n[16.939098, \"o\", \" \"]\n[17.6128, \"o\", \"n\"]\n[17.666004, \"o\", \"e\"]\n[17.810336, \"o\", \"w\"]\n[17.917136, \"o\", \"-\"]\n[18.016204, \"o\", \"f\"]\n[18.101805, \"o\", \"i\"]\n[18.250022, \"o\", \"l\"]\n[18.312769, \"o\", \"e\"]\n[18.473484, \"o\", \"\\u001b[?1l\\u001b>\"]\n[18.482016, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[18.484593, \"o\", \"\\u001b]2;nvim new-file\\u0007\"]\n[18.485713, \"o\", \"\\u001b]1;vi\\u0007\"]\n[18.692978, \"o\", \"\\u001b[?1049h\\u001b[22;0;0t\\u001b[?1h\\u001b=\\u001b[H\\u001b[2J\\u001b[?2004h\\u001b[?1004h\\u001b[8;29;119t\\u001b[r\\u001b[1;1H\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40m \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \"]\n[18.693131, \"o\", \" \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \"]\n[18.693209, \"o\", \" \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \"]\n[18.693313, \"o\", \" \\r\\n \\r\\n \\r\\n \\r\\n \\u001b[H\\u001b[?25h\"]\n[18.693852, \"o\", \"\\u001b[?25l\\u001b[2 q\\u001b[2 q \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \"]\n[18.69393, \"o\", \" \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \"]\n[18.693987, \"o\", \" \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \\r\\n \"]\n[18.694043, \"o\", \" \\r\\n \\r\\n \\r\\n \\u001b[H\\u001b[28B\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40m\\\"new-file\\\"\\u001b[C[Neue\\u001b[CDatei]\\u001b[?25h\"]\n[18.755878, \"o\", \"\\u001b[?25l\\u001b[H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;254;128;25m buffers \\u001b(B\\u001b[m\\u001b[38;2;254;128;25m\\u001b[48;2;168;153;132m \\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;168;153;132m new-file \\u001b(B\\u001b[m\\u001b[38;2;168;153;132m\\u001b[48;2;40;40;40m \\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;40;40;40m ◀\\u001b(B\\u001b[m\\u001b[38;2;168;153;132m\\u001b[48;2;40;40;40m◀\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;168;153;132m [new-file] \\u001b(B\\u001b[m\\u001b[38;2;254;128;25m\\u001b[48;2;168;153;132m◀\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;254;128;25m tabs \\r\\n\\u001b(B\\u001b[m\\u001b[38;2;124;111;100m\\u001b[48;2;40;40;40m 1 \\r\\n\\u001b(B\\u001b[m\\u001b[38;2;80;73;69m\\u001b[48;2;40;40;40m~ \\r\\n~ \\r\\n~ \\r\\n~ \"]\n[18.756006, \"o\", \" \\r\\n~ \\r\\n~ \\r\\n~ \\r\\n~ \\r\\n~ \\r\\n~ \\r\\n~ \\r\\n~ \\r\\n~ \"]\n[18.756032, \"o\", \" \\r\\n~ \\r\\n~ \\r\\n~ \\r\\n~ \\r\\n~ \\r\\n~ \\r\\n~ \\r\\n~ \"]\n[18.756049, \"o\", \" \\r\\n~ \\r\\n~ \\r\\n~ \\r\\n~ \\r\\n\\u001b[?25h\"]\n[18.760143, \"o\", \"\\u001b[?25l\"]\n[18.776267, \"o\", \"\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;168;153;132m \\u001b(B\\u001b[0;1m\\u001b[38;2;40;40;40m\\u001b[48;2;168;153;132mNORMAL\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;168;153;132m » SPELL [HUN-DE-DE-FRAMI] \\u001b(B\\u001b[m\\u001b[38;2;168;153;132m\\u001b[48;2;80;73;69m▶\\u001b(B\\u001b[m\\u001b[38;2;80;73;69m\\u001b[48;2;60;56;54m▶\\u001b(B\\u001b[m\\u001b[38;2;168;153;132m\\u001b[48;2;60;56;54m new-file \\u001b(B\\u001b[m\\u001b[38;2;60;56;54m\\u001b[48;2;60;56;54m◀\\u001b(B\\u001b[m\\u001b[38;2;80;73;69m\\u001b[48;2;60;56;54m◀\\u001b(B\\u001b[m\\u001b[38;2;168;153;132m\\u001b[48;2;80;73;69m utf-8[BOM][unix] ◀\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;168;153;132m 100% \\u001b(B\\u001b[0;1m\\u001b[38;2;40;40;40m\\u001b[48;2;168;153;132m␊ 0/1 ㏑\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;168;153;132m : 1 \\u001b(B\\u001b[m\\u001b[38;2;254;128;25m\\u001b[48;2;168;153;132m◀\\u001b(B\\u001b[m\\u001b[38;2;251;73;52m\\u001b[48;2;254;128;25m◀\\r\\n\\u001b]0;/tmp/mount/new-file\\u0007\\u001b[?25h\"]\n[18.77748, \"o\", \"\\u001b[?25l\\u001b[2;5H\\u001b[?25h\"]\n[19.345027, \"o\", \"\\u001b[?25l\\u001b[29;109H\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mi\\u001b[2;5H\\u001b[?25h\"]\n[19.345442, \"o\", \"\\u001b[?25l\\u001b[6 q\\u001b[29;109H \\u001b[2;5H\\u001b[?25h\"]\n[19.346299, \"o\", \"\\u001b[?25l\\r\\u001b[27B\\u001b(B\\u001b[0;1m\\u001b[38;2;250;189;47m\\u001b[48;2;40;40;40m-- EINFÜGEN --\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40m \\u001b[29;15H\\u001b[?25h\"]\n[19.377501, \"o\", \"\\u001b[?25l\\r\\u001b[A\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m \\u001b(B\\u001b[0;1m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152mINSERT\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m » SPELL [HUN-DE-DE-FRAMI] \\u001b(B\\u001b[m\\u001b[38;2;131;165;152m\\u001b[48;2;80;73;69m▶\\u001b(B\\u001b[m\\u001b[38;2;80;73;69m\\u001b[48;2;80;73;69m▶\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;80;73;69m new-file \\u001b(B\\u001b[m\\u001b[38;2;80;73;69m\\u001b[48;2;80;73;69m◀◀\\u001b[18C\\u001b(B\\u001b[m\\u001b[38;2;131;165;152m\\u001b[48;2;80;73;69m◀\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m 100% \\u001b(B\\u001b[0;1m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m␊ 0/1 ㏑\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m : 1 \\u001b(B\\u001b[m\\u001b[38;2;254;128;25m\\u001b[48;2;131;165;152m◀\\u001b[2;5H\\u001b[?25h\"]\n[19.901032, \"o\", \"\\u001b[?25l\\u001b[1;20H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;168;153;132m+ \\u001b(B\\u001b[m\\u001b[38;2;168;153;132m\\u001b[48;2;40;40;40m \\u001b[2;5H\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mT\\u001b[28;37H\\u001b(B\\u001b[m\\u001b[38;2;131;165;152m\\u001b[48;2;80;73;69m new-file[+] \\u001b[31C\\u001b(B\\u001b[0;1m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m1\\u001b[9C\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m2\\u001b[2;6H\\u001b[?25h\"]\n[19.905544, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mh\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m3\\u001b[2;7H\\u001b[?25h\"]\n[20.080393, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mi\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m4\\u001b[2;8H\\u001b[?25h\"]\n[20.164795, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40ms\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m5\\u001b[2;9H\\u001b[?25h\"]\n[20.258114, \"o\", \"\\u001b[?25l\\b\\b\\b\\b\\u001b(B\\u001b[0;4m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mThis\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m6\\u001b[2;10H\\u001b[?25h\"]\n[20.673937, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mi\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m7\\u001b[2;11H\\u001b[?25h\"]\n[21.163759, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40ms\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m8\\u001b[2;12H\\u001b[?25h\"]\n[21.317033, \"o\", \"\\u001b[?25l\\b\\b\\u001b(B\\u001b[0;4m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mis\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m9\\u001b[2;13H\\u001b[?25h\"]\n[21.403983, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40ma\\u001b[28;115H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m10\\u001b[2;14H\\u001b[?25h\"]\n[21.515109, \"o\", \"\\u001b[?25l\\u001b[28;116H1\\u001b[2;15H\\u001b[?25h\"]\n[21.579911, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mf\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m2\\u001b[2;16H\\u001b[?25h\"]\n[21.919128, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mn\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m3\\u001b[2;17H\\u001b[?25h\"]\n[21.959223, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40me\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m4\\u001b[2;18H\\u001b[?25h\"]\n[22.191207, \"o\", \"\\u001b[?25l\\u001b[2 q\\u001b[6 q\\u001b[?25h\"]\n[22.205546, \"o\", \"\\u001b[?25l\\b\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40m \\u001b[2;17H\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m3\\u001b[2;17H\\u001b[?25h\"]\n[22.349697, \"o\", \"\\u001b[?25l\\u001b[2 q\\u001b[6 q\\u001b[?25h\"]\n[22.359876, \"o\", \"\\u001b[?25l\\b\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40m \\u001b[2;16H\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m2\\u001b[2;16H\\u001b[?25h\"]\n[22.486515, \"o\", \"\\u001b[?25l\\u001b[2 q\\u001b[6 q\\u001b[?25h\"]\n[22.497949, \"o\", \"\\u001b[?25l\\b\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40m \\u001b[2;15H\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m1\\u001b[2;15H\\u001b[?25h\"]\n[22.683688, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mn\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m2\\u001b[2;16H\\u001b[?25h\"]\n[22.738237, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40me\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m3\\u001b[2;17H\\u001b[?25h\"]\n[22.890398, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mw\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m4\\u001b[2;18H\\u001b[?25h\"]\n[23.014604, \"o\", \"\\u001b[?25l\\b\\b\\b\\u001b(B\\u001b[0;4m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mnew\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m5\\u001b[2;19H\\u001b[?25h\"]\n[23.051557, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mf\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m6\\u001b[2;20H\\u001b[?25h\"]\n[23.166211, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mi\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m7\\u001b[2;21H\\u001b[?25h\"]\n[23.304776, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40ml\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m8\\u001b[2;22H\\u001b[?25h\"]\n[23.324634, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40me\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m9\\u001b[2;23H\\u001b[?25h\"]\n[23.404199, \"o\", \"\\u001b[?25l\\b\\b\\b\\b\\u001b(B\\u001b[0;4m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mfile\\u001b[28;115H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m20\\u001b[2;24H\\u001b[?25h\"]\n[23.523839, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40ma\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m1\\u001b[2;25H\\u001b[?25h\"]\n[23.647926, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mdn\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m3\\u001b[2;27H\\u001b[?25h\"]\n[23.865008, \"o\", \"\\u001b[?25l\\b\\b\\b\\u001b(B\\u001b[0;4m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40madn\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m4\\u001b[2;28H\\u001b[?25h\"]\n[24.140954, \"o\", \"\\u001b[?25l\\u001b[2 q\\u001b[6 q\\u001b[?25h\"]\n[24.158755, \"o\", \"\\u001b[?25l\\b\\b\\b\\b\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40madn\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m3\\u001b[2;27H\\u001b[?25h\"]\n[24.294178, \"o\", \"\\u001b[?25l\\u001b[2 q\\u001b[6 q\\u001b[?25h\"]\n[24.30896, \"o\", \"\\u001b[?25l\"]\n[24.309298, \"o\", \"\\b\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40m \\u001b[2;26H\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m2\\u001b[2;26H\"]\n[24.309498, \"o\", \"\\u001b[?25h\"]\n[24.429442, \"o\", \"\\u001b[?25l\\u001b[2 q\\u001b[6 q\\u001b[?25h\"]\n[24.448041, \"o\", \"\\u001b[?25l\\b\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40m \\u001b[2;25H\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m1\\u001b[2;25H\\u001b[?25h\"]\n[24.621783, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mn\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m2\\u001b[2;26H\\u001b[?25h\"]\n[24.69289, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40md\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m3\\u001b[2;27H\\u001b[?25h\"]\n[24.75518, \"o\", \"\\u001b[?25l\\b\\b\\b\\u001b(B\\u001b[0;4m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mand\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m4\\u001b[2;28H\\u001b[?25h\"]\n[24.882117, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mI\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m5\\u001b[2;29H\\u001b[?25h\"]\n[24.990957, \"o\", \"\\u001b[?25l\\u001b[28;116H6\\u001b[2;30H\\u001b[?25h\"]\n[25.147616, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mc\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m7\\u001b[2;31H\\u001b[?25h\"]\n[25.259868, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40ma\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m8\\u001b[2;32H\\u001b[?25h\"]\n[25.338547, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mn\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m9\\u001b[2;33H\\u001b[?25h\"]\n[25.563801, \"o\", \"\\u001b[?25l\\b\\b\\b\\u001b(B\\u001b[0;4m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mcan\\u001b[28;115H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m30\\u001b[2;34H\\u001b[?25h\"]\n[25.781546, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mu\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m1\\u001b[2;35H\\u001b[?25h\"]\n[25.811621, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40ms\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m2\\u001b[2;36H\\u001b[?25h\"]\n[25.971726, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40me\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m3\\u001b[2;37H\\u001b[?25h\"]\n[26.056187, \"o\", \"\\u001b[?25l\\b\\b\\b\\u001b(B\\u001b[0;4m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40muse\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m4\\u001b[2;38H\\u001b[?25h\"]\n[26.157084, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mt\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m5\\u001b[2;39H\\u001b[?25h\"]\n[26.216725, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mh\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m6\\u001b[2;40H\\u001b[?25h\"]\n[26.33611, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mi\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m7\\u001b[2;41H\\u001b[?25h\"]\n[26.402646, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40ms\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m8\\u001b[2;42H\\u001b[?25h\"]\n[26.469983, \"o\", \"\\u001b[?25l\\b\\b\\b\\b\\u001b(B\\u001b[0;4m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mthis\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m9\\u001b[2;43H\\u001b[?25h\"]\n[26.604867, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mf\\u001b[28;115H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m40\\u001b[2;44H\\u001b[?25h\"]\n[26.688028, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mo\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m1\\u001b[2;45H\\u001b[?25h\"]\n[26.831965, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40ml\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m2\\u001b[2;46H\\u001b[?25h\"]\n[26.876067, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40md\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m3\\u001b[2;47H\\u001b[?25h\"]\n[26.960031, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40me\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m4\\u001b[2;48H\\u001b[?25h\"]\n[27.028809, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mr\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m5\\u001b[2;49H\\u001b[?25h\"]\n[27.101686, \"o\", \"\\u001b[?25l\\u001b[6D\\u001b(B\\u001b[0;4m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mfolder\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m6\\u001b[2;50H\\u001b[?25h\"]\n[27.345086, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mj\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m7\\u001b[2;51H\\u001b[?25h\"]\n[27.524854, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mu\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m8\\u001b[2;52H\\u001b[?25h\"]\n[27.608849, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40ms\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m9\\u001b[2;53H\\u001b[?25h\"]\n[27.732593, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mt\\u001b[28;115H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m50\\u001b[2;54H\\u001b[?25h\"]\n[27.818347, \"o\", \"\\u001b[?25l\\u001b[28;116H1\\u001b[2;55H\\u001b[?25h\"]\n[27.896692, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40ml\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m2\\u001b[2;56H\\u001b[?25h\"]\n[28.032423, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mi\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m3\\u001b[2;57H\\u001b[?25h\"]\n[28.1877, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mk\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m4\\u001b[2;58H\\u001b[?25h\"]\n[28.263534, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40me\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m5\\u001b[2;59H\\u001b[?25h\"]\n[28.355656, \"o\", \"\\u001b[?25l\\b\\b\\b\\b\\u001b(B\\u001b[0;4m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mlike\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m6\\u001b[2;60H\\u001b[?25h\"]\n[28.484556, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40ma\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m7\\u001b[2;61H\\u001b[?25h\"]\n[28.565738, \"o\", \"\\u001b[?25l\\u001b[28;116H8\\u001b[2;62H\\u001b[?25h\"]\n[28.672064, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mn\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m9\\u001b[2;63H\\u001b[?25h\"]\n[28.866181, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mo\\u001b[28;115H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m60\\u001b[2;64H\\u001b[?25h\"]\n[28.935765, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mr\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m1\\u001b[2;65H\\u001b[?25h\"]\n[29.071917, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mm\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m2\\u001b[2;66H\\u001b[?25h\"]\n[29.189156, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40ma\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m3\\u001b[2;67H\\u001b[?25h\"]\n[29.254533, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40ml\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m4\\u001b[2;68H\\u001b[?25h\"]\n[29.345343, \"o\", \"\\u001b[?25l\\u001b[28;116H5\\u001b[2;69H\\u001b[?25h\"]\n[30.022506, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mf\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m6\\u001b[2;70H\\u001b[?25h\"]\n[30.09986, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mo\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m7\\u001b[2;71H\\u001b[?25h\"]\n[30.266437, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40ml\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m8\\u001b[2;72H\\u001b[?25h\"]\n[30.290487, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40md\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m9\\u001b[2;73H\\u001b[?25h\"]\n[30.393706, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40me\\u001b[28;115H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m70\\u001b[2;74H\\u001b[?25h\"]\n[30.451108, \"o\", \"\\u001b[?25l\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mr\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m1\\u001b[2;75H\\u001b[?25h\"]\n[30.645571, \"o\", \"\\u001b[?25l\\u001b[6D\\u001b(B\\u001b[0;4m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40mfolder\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40m.\\u001b[28;116H\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;131;165;152m2\\u001b[2;76H\\u001b[?25h\"]\n[30.860939, \"o\", \"\\u001b[?25l\\r\\u001b[27B\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40m \\r\\u001b[29;1H\\u001b[2 q\\u001b[?25h\"]\n[30.915162, \"o\", \"\\u001b[?25l\\u001b[A\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;168;153;132m \\u001b(B\\u001b[0;1m\\u001b[38;2;40;40;40m\\u001b[48;2;168;153;132mNORMAL\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;168;153;132m » SPELL [HUN-DE-DE-FRAMI] \\u001b(B\\u001b[m\\u001b[38;2;168;153;132m\\u001b[48;2;80;73;69m▶\\u001b(B\\u001b[m\\u001b[38;2;80;73;69m\\u001b[48;2;60;56;54m▶\\u001b(B\\u001b[m\\u001b[38;2;131;165;152m\\u001b[48;2;60;56;54m new-file[+] \\u001b(B\\u001b[m\\u001b[38;2;60;56;54m\\u001b[48;2;60;56;54m◀\\u001b(B\\u001b[m\\u001b[38;2;80;73;69m\\u001b[48;2;60;56;54m◀\\u001b[18C\\u001b(B\\u001b[m\\u001b[38;2;168;153;132m\\u001b[48;2;80;73;69m◀\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;168;153;132m 100% \\u001b(B\\u001b[0;1m\\u001b[38;2;40;40;40m\\u001b[48;2;168;153;132m␊ 1/1 ㏑\\u001b(B\\u001b[m\\u001b[38;2;40;40;40m\\u001b[48;2;168;153;132m : 71 \\u001b(B\\u001b[m\\u001b[38;2;254;128;25m\\u001b[48;2;168;153;132m◀\\u001b[2;75H\\u001b[?25h\"]\n[31.136268, \"o\", \"\\u001b[?25l\\u001b[29;109H\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40m:\\u001b[2;75H\\u001b[?25h\"]\n[31.136405, \"o\", \"\\u001b[?25l\\u001b[29;109H\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40m \\u001b[29;109H\\r\\u001b[29;1H\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40m:\\u001b[2 q\\u001b[?25h\"]\n[31.224948, \"o\", \"\\u001b[?25lw\\u001b[?25h\"]\n[31.286773, \"o\", \"\\u001b[?25lq\\u001b[?25h\"]\n[31.42495, \"o\", \"\\u001b[?25l\\r\\u001b[29;1H\\u001b[?25h\"]\n[31.427199, \"o\", \"\\u001b[?25l\\u001b[2 q\\\"new-file\\\"\\u001b[?25h\"]\n[31.670283, \"o\", \"\\u001b[?25l\\u001b[C[Neu]\\u001b[C1L,\\u001b[C72C\\u001b[Cgeschrieben\\u001b[?25h\"]\n[31.677007, \"o\", \"\\u001b[?25l\"]\n[31.6881, \"o\", \"\\u001b[?25h\"]\n[31.717445, \"o\", \"\\u001b[?25l\\r\\u001b[29;1H\\u001b[H\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40m\\u001b[M\\u001b[28B\\u001b(B\\u001b[0;1m\\u001b[38;2;40;40;40m\\u001b[48;2;251;73;52mFehler beim Ausführen von \\\"VimLeave Autokommandos für \\\"*\\\"\\\":\\u001b[?25h\\u001b[?25l\\u001b[H\\u001b(B\\u001b[m\\u001b[38;2;235;219;178m\\u001b[48;2;40;40;40m\\u001b[M\\u001b[29;60H\\r\\u001b[29;1H\\u001b(B\\u001b[0;1m\\u001b[38;2;40;40;40m\\u001b[48;2;251;73;52mE488: Überschüssige Zeichen\\u001b[?25h\"]\n[31.717957, \"o\", \"\\u001b[?25l\\r\\u001b[29;1H\\u001b[2 q\\u001b(B\\u001b[m\\u001b[?25h\\u001b[?1l\\u001b>\\u001b[?1049l\\u001b[23;0;0t\\u001b[?2004l\\u001b[?1004l\\u001b[?25h\"]\n[31.799109, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\\u001b]2;sahib@werkbank: /tmp/mount\\u0007\\u001b]1;/tmp/mount\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\\u001b[?1h\\u001b=\\u001b[?2004h\"]\n[33.189103, \"o\", \"\\u001b[32ml\\u001b[39m\"]\n[33.316672, \"o\", \"\\b\\u001b[32ml\\u001b[32ms\\u001b[39m\"]\n[33.618794, \"o\", \"\\u001b[?1l\\u001b>\"]\n[33.622899, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[33.625006, \"o\", \"\\u001b]2;ls --color=tty\\u0007\\u001b]1;ls\\u0007\"]\n[33.634291, \"o\", \"\\u001b[0m\\u001b[35mnew-file\\u001b[0m \\u001b[35mREADME.md\\u001b[0m \\u001b[1msub\\u001b[0m\\r\\n\"]\n[33.635036, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[33.63557, \"o\", \"\\u001b]2;sahib@werkbank: /tmp/mount\\u0007\"]\n[33.635961, \"o\", \"\\u001b]1;/tmp/mount\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[33.636845, \"o\", \"\\u001b[?1h\\u001b=\"]\n[33.637693, \"o\", \"\\u001b[?2004h\"]\n[34.15552, \"o\", \"\\u001b[1m\\u001b[31mc\\u001b[0m\\u001b[39m\"]\n[34.286797, \"o\", \"\\b\\u001b[1m\\u001b[31mc\\u001b[1m\\u001b[31ma\\u001b[0m\\u001b[39m\"]\n[34.455531, \"o\", \"\\b\\b\\u001b[0m\\u001b[32mc\\u001b[0m\\u001b[32ma\\u001b[32mt\\u001b[39m\"]\n[34.566448, \"o\", \" \"]\n[35.046696, \"o\", \"\\u001b[4mn\\u001b[24m\"]\n[35.112191, \"o\", \"\\b\\u001b[4mn\\u001b[4me\\u001b[24m\"]\n[35.273926, \"o\", \"\\b\\u001b[4me\\u001b[4mw\\u001b[24m\"]\n[35.414208, \"o\", \"\\b\\u001b[4mw\\u001b[4m-\\u001b[24m\"]\n[35.529747, \"o\", \"\\b\\u001b[4m-\\u001b[4mf\\u001b[24m\"]\n[35.627323, \"o\", \"\\b\\u001b[4mf\\u001b[4mi\\u001b[24m\"]\n[35.779769, \"o\", \"\\b\\u001b[4mi\\u001b[4ml\\u001b[24m\"]\n[35.883377, \"o\", \"\\b\\u001b[4ml\\u001b[4me\\u001b[24m\"]\n[36.375857, \"o\", \"\\u001b[?1l\\u001b>\"]\n[36.381644, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[36.383975, \"o\", \"\\u001b]2;cat new-file\\u0007\\u001b]1;cat\\u0007\"]\n[36.402338, \"o\", \"This is a new file and I can use this folder just like a normal folder.\\r\\n\"]\n[36.407302, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[36.408527, \"o\", \"\\u001b]2;sahib@werkbank: /tmp/mount\\u0007\\u001b]1;/tmp/mount\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[36.40922, \"o\", \"\\u001b[?1h\\u001b=\"]\n[36.411242, \"o\", \"\\u001b[?2004h\"]\n[37.968678, \"o\", \"\\u001b[1m\\u001b[31mm\\u001b[0m\\u001b[39m\"]\n[38.166901, \"o\", \"\\b\\u001b[1m\\u001b[31mm\\u001b[1m\\u001b[31mp\\u001b[0m\\u001b[39m\"]\n[38.246176, \"o\", \"\\b\\b\\u001b[0m\\u001b[32mm\\u001b[0m\\u001b[32mp\\u001b[32mv\\u001b[39m\"]\n[38.350516, \"o\", \" \"]\n[38.465578, \"o\", \"\\u001b[4ms\\u001b[24m\"]\n[38.566672, \"o\", \"\\b\\u001b[4ms\\u001b[4mu\\u001b[24m\"]\n[38.690564, \"o\", \"\\b\\u001b[4mu\\u001b[4mb\\u001b[24m\"]\n[38.876413, \"o\", \"\\b\\u001b[4mb\\u001b[4m/\\u001b[24m\"]\n[39.302298, \"o\", \"\\b\\u001b[4m/\\u001b[4mmusic.mp3\\u001b[24m \"]\n[39.993006, \"o\", \"\\u001b[?1l\\u001b>\"]\n[39.998325, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[40.000757, \"o\", \"\\u001b]2;mpv sub/music.mp3\\u0007\\u001b]1;mpv\\u0007\"]\n[40.280961, \"o\", \"\\u001b=\"]\n[40.286502, \"o\", \"\\u001b[0mPlaying: sub/music.mp3\\r\\n\\u001b[0m\"]\n[40.30279, \"o\", \"\\u001b[0;33m[ffmpeg/demuxer] mp3: invalid concatenated file detected - using bitrate for duration\\r\\n\\u001b[0m\"]\n[40.303922, \"o\", \"\\u001b[0;33m[ffmpeg/demuxer] mp3: Estimating duration from bitrate, this may be inaccurate\\r\\n\\u001b[0m\"]\n[40.306491, \"o\", \"\\u001b[0m (+) Audio --aid=1 (mp3 2ch 44100Hz)\\r\\n\\u001b[0m\\u001b[0mFile tags:\\r\\n\\u001b[0m\"]\n[40.306638, \"o\", \"\\u001b[0m Artist: Epica\\r\\n\\u001b[0m\\u001b[0m Album: The Classical Conspiracy: Live in Miskolc, Hungary\\r\\n\\u001b[0m\\u001b[0m Album_Artist: Epica\\r\\n\\u001b[0m\\u001b[0m Title: The Imperial March\\r\\n\\u001b[0m\\u001b[0m Track: 8/18\\r\\n\\u001b[0m\"]\n[40.310502, \"o\", \"\\u001b[0mAO: [pulse] 44100Hz stereo 2ch s16\\r\\n\\u001b[0m\"]\n[40.336663, \"o\", \"\\r\\u001b[K\\u001b[0mA: 00:00:00 / 00:06:50 (0%) Cache: 10s+1MB\\r\\u001b[0m\"]\n[40.450885, \"o\", \"\\r\\u001b[K\\u001b[0mA: 00:00:00 / 00:06:50 (0%) Cache: 10s+6MB\\r\\u001b[0m\"]\n[40.525866, \"o\", \"\\r\\u001b[K\\u001b[0m\"]\n[40.525925, \"o\", \"A: 00:00:00 / 00:06:50 (0%) Cache: 10s+11MB\\r\\u001b[0m\"]\n[40.575935, \"o\", \"\\r\\u001b[K\\u001b[0mA: 00:00:00 / 00:06:50 (0%) Cache: 10s+12MB\\r\\u001b[0m\"]\n[41.337249, \"o\", \"\\r\\u001b[K\\u001b[0mA: 00:00:01 / 00:06:50 (0%) Cache: 10s+12MB\\r\\u001b[0m\"]\n[42.315624, \"o\", \"\\r\\u001b[K\\u001b[0mA: 00:00:02 / 00:06:50 (0%) Cache: 10s+12MB\\r\\u001b[0m\"]\n[43.335133, \"o\", \"\\r\\u001b[K\\u001b[0mA: 00:00:03 / 00:06:50 (0%) Cache: 10s+12MB\\r\\u001b[0m\"]\n[44.356615, \"o\", \"\\r\\u001b[K\\u001b[0mA: 00:00:04 / 00:06:50 (0%) Cache: 10s+12MB\\r\\u001b[0m\"]\n[44.42698, \"o\", \"\\r\\u001b[K\\u001b[0mA: 00:00:04 / 00:06:50 (1%) Cache: 10s+12MB\\r\\u001b[0m\"]\n[44.561352, \"o\", \"\\r\\u001b[K\\u001b[0m\"]\n[44.56157, \"o\", \"A: 00:00:04 / 00:06:50 (1%) Cache: 9s+12MB\\r\\u001b[0m\"]\n[44.631238, \"o\", \"\\r\\u001b[K\\u001b[0mA: 00:00:04 / 00:06:50 (1%) Cache: 10s+12MB\\r\\u001b[0m\"]\n[44.790759, \"o\", \"\\u001b>\"]\n[44.801467, \"o\", \"\\r\\n\\u001b[0m\\r\\n\\u001b[0m\\u001b[0m\\r\\n\\u001b[0m\\u001b[0mExiting... (Quit)\\r\\n\\u001b[0m\"]\n[44.82915, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[44.829887, \"o\", \"\\u001b]2;sahib@werkbank: /tmp/mount\\u0007\\u001b]1;/tmp/mount\\u0007\"]\n[44.830358, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[44.830797, \"o\", \"\\u001b[?1h\\u001b=\"]\n[44.831683, \"o\", \"\\u001b[?2004h\"]\n[45.582474, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[45.738395, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[45.836404, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[45.920141, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[45.992154, \"o\", \" \"]\n[46.078312, \"o\", \"l\"]\n[46.169958, \"o\", \"s\"]\n[46.259206, \"o\", \"\\u001b[?1l\\u001b>\"]\n[46.267116, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[46.269603, \"o\", \"\\u001b]2;brig ls\\u0007\"]\n[46.270458, \"o\", \"\\u001b]1;brig\\u0007\"]\n[46.466591, \"o\", \"SIZE MODTIME PATH PIN \\r\\n886 B Feb 28 10:45:21 \\u001b[37m/README.md\\u001b[0m \\u001b[36m🖈\\u001b[0m \\r\\n72 B Feb 28 10:48:51 \\u001b[37m/new-file\\u001b[0m \\u001b[36m🖈\\u001b[0m \\r\\n\\u001b[33m13 MB\\u001b[0m Feb 28 10:46:48\"]\n[46.466663, \"o\", \" \\u001b[32m/sub\\u001b[0m \\u001b[36m🖈\\u001b[0m \\r\\n\"]\n[46.468235, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[46.468348, \"o\", \"\\u001b]2;sahib@werkbank: /tmp/mount\\u0007\"]\n[46.468429, \"o\", \"\\u001b]1;/tmp/mount\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[46.468519, \"o\", \"\\u001b[?1h\\u001b=\"]\n[46.468726, \"o\", \"\\u001b[?2004h\"]\n" }, { "path": "docs/asciinema/5_commits.json", "content": "{\"version\": 2, \"width\": 119, \"height\": 29, \"timestamp\": 1519811965, \"env\": {\"SHELL\": \"/bin/zsh\", \"TERM\": \"xterm-256color\"}}\n[0.252493, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[0.252647, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[0.252698, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[0.252849, \"o\", \"\\u001b[?1h\\u001b=\"]\n[0.253042, \"o\", \"\\u001b[?2004h\"]\n[1.089292, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[1.199596, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[1.295245, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[1.394617, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[1.475068, \"o\", \" \"]\n[1.532962, \"o\", \"\\u001b[4ml\\u001b[24m\"]\n[1.689091, \"o\", \"\\b\\u001b[4ml\\u001b[4mo\\u001b[24m\"]\n[1.764881, \"o\", \"\\b\\u001b[4mo\\u001b[4mg\\u001b[24m\"]\n[2.164452, \"o\", \"\\u001b[?1l\\u001b>\"]\n[2.168482, \"o\", \"\\u001b[?2004l\\b\\b\\b\\u001b[24ml\\u001b[24mo\\u001b[24mg\\r\\r\\n\"]\n[2.171031, \"o\", \"\\u001b]2;brig log\\u0007\"]\n[2.17135, \"o\", \"\\u001b]1;brig\\u0007\"]\n[2.32099, \"o\", \"\\u001b[32mSEfXUD64dE\\u001b[0m \\u001b[33mFeb 28 10:45:21\\u001b[0m \\u001b[31m•\\u001b[0m\\u001b[36m (curr)\\u001b[0m\\r\\n\"]\n[2.32113, \"o\", \"\\u001b[32mSEfXUCU47p\\u001b[0m \\u001b[33mFeb 28 10:45:21\\u001b[0m Added initial README.md\\u001b[36m (head)\\u001b[0m\\r\\n\\u001b[32mSEfXUCEaXL\\u001b[0m \\u001b[33mFeb 28 10:45:21\\u001b[0m initial commit\\u001b[36m (init)\\u001b[0m\\r\\n\"]\n[2.322517, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[2.322672, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[2.322731, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[2.322798, \"o\", \"\\u001b[?1h\\u001b=\"]\n[2.323008, \"o\", \"\\u001b[?2004h\"]\n[3.60464, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[3.701166, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[3.806198, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[3.908073, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[3.981747, \"o\", \" \"]\n[4.083177, \"o\", \"\\u001b[4mc\\u001b[24m\"]\n[4.180388, \"o\", \"\\b\\u001b[4mc\\u001b[4mo\\u001b[24m\"]\n[4.34891, \"o\", \"\\b\\u001b[4mo\\u001b[4mm\\u001b[24m\"]\n[4.471983, \"o\", \"\\b\\b\\b\\u001b[24mc\\u001b[24mo\\u001b[24mmm\"]\n[4.662754, \"o\", \"i\"]\n[4.794937, \"o\", \"t\"]\n[4.828483, \"o\", \" \"]\n[4.91971, \"o\", \"-\"]\n[5.12147, \"o\", \"m\"]\n[5.232306, \"o\", \" \"]\n[5.36732, \"o\", \"\\u001b[33m'\\u001b[39m\"]\n[6.367657, \"o\", \"\\b\\u001b[33m'\\u001b[33mA\\u001b[39m\"]\n[6.542322, \"o\", \"\\b\\u001b[33mA\\u001b[33md\\u001b[39m\"]\n[6.703172, \"o\", \"\\b\\u001b[33md\\u001b[33md\\u001b[39m\"]\n[6.98262, \"o\", \"\\b\\u001b[33md\\u001b[33me\\u001b[39m\"]\n[7.114576, \"o\", \"\\b\\u001b[33me\\u001b[33md\\u001b[39m\"]\n[7.226658, \"o\", \"\\b\\u001b[33md\\u001b[33m \\u001b[39m\"]\n[7.400424, \"o\", \"\\b\\u001b[33m \\u001b[33md\\u001b[39m\"]\n[7.48366, \"o\", \"\\b\\u001b[33md\\u001b[33ma\\u001b[39m\"]\n[7.601921, \"o\", \"\\b\\u001b[33ma\\u001b[33mr\\u001b[39m\"]\n[7.741708, \"o\", \"\\b\\u001b[33mr\\u001b[33mt\\u001b[39m\"]\n[7.830802, \"o\", \"\\b\\u001b[33mt\\u001b[33mh\\u001b[39m\"]\n[7.915606, \"o\", \"\\b\\u001b[33mh\\u001b[33m \\u001b[39m\"]\n[8.040754, \"o\", \"\\b\\u001b[33m \\u001b[33mv\\u001b[39m\"]\n[8.142824, \"o\", \"\\b\\u001b[33mv\\u001b[33ma\\u001b[39m\"]\n[8.275411, \"o\", \"\\b\\u001b[33ma\\u001b[33md\\u001b[39m\"]\n[8.392729, \"o\", \"\\b\\u001b[33md\\u001b[33me\\u001b[39m\"]\n[8.481706, \"o\", \"\\b\\u001b[33me\\u001b[33mr\\u001b[39m\"]\n[8.867549, \"o\", \"\\b\\u001b[33mr\\u001b[33m \\u001b[39m\"]\n[9.017732, \"o\", \"\\b\\u001b[33m \\u001b[33mm\\u001b[39m\"]\n[9.201591, \"o\", \"\\b\\u001b[33mm\\u001b[33mu\\u001b[39m\"]\n[9.306462, \"o\", \"\\b\\u001b[33mu\\u001b[33ms\\u001b[39m\"]\n[9.421788, \"o\", \"\\b\\u001b[33ms\\u001b[33mi\\u001b[39m\"]\n[9.482797, \"o\", \"\\b\\u001b[33mi\\u001b[33mc\\u001b[39m\"]\n[9.746346, \"o\", \"\\b\\u001b[33mc\\u001b[33m'\\u001b[39m\"]\n[10.36415, \"o\", \"\\u001b[?1l\\u001b>\"]\n[10.370844, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[10.374473, \"o\", \"\\u001b]2;brig commit -m 'Added darth vader music'\\u0007\\u001b]1;brig\\u0007\"]\n[10.562116, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[10.562253, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[10.562287, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[10.562385, \"o\", \"\\u001b[?1h\\u001b=\"]\n[10.562584, \"o\", \"\\u001b[?2004h\"]\n[11.482654, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[11.595748, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[11.713421, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[11.801969, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[11.871499, \"o\", \" \"]\n[11.986051, \"o\", \"\\u001b[4ml\\u001b[24m\"]\n[12.130566, \"o\", \"\\b\\u001b[4ml\\u001b[4mo\\u001b[24m\"]\n[12.208039, \"o\", \"\\b\\u001b[4mo\\u001b[4mg\\u001b[24m\"]\n[12.3702, \"o\", \"\\u001b[?1l\\u001b>\"]\n[12.375073, \"o\", \"\\u001b[?2004l\\b\\b\\b\\u001b[24ml\\u001b[24mo\\u001b[24mg\\r\\r\\n\"]\n[12.377147, \"o\", \"\\u001b]2;brig log\\u0007\\u001b]1;brig\\u0007\"]\n[12.483055, \"o\", \"\\u001b[32mSEfXUBgTGT\\u001b[0m \\u001b[33mFeb 28 10:59:36\\u001b[0m \\u001b[31m•\\u001b[0m\\u001b[36m (curr)\\u001b[0m\\r\\n\"]\n[12.483199, \"o\", \"\\u001b[32mSEfXUCjYxA\\u001b[0m \\u001b[33mFeb 28 10:45:21\\u001b[0m Added darth vader music\\u001b[36m (head)\\u001b[0m\\r\\n\\u001b[32mSEfXUCU47p\\u001b[0m \\u001b[33mFeb 28 10:45:21\\u001b[0m Added initial README.md\\u001b[36m\\u001b[0m\\r\\n\\u001b[32mSEfXUCEaXL\\u001b[0m \\u001b[33mFeb 28 10:45:21\\u001b[0m initial commit\\u001b[36m (init)\\u001b[0m\\r\\n\"]\n[12.484754, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[12.484868, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\"]\n[12.484981, \"o\", \"\\u001b]1;~/dev/brig/docs\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[12.485059, \"o\", \"\\u001b[?1h\\u001b=\"]\n[12.48524, \"o\", \"\\u001b[?2004h\"]\n[15.116498, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[15.286419, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[15.394661, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[15.484396, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[15.570743, \"o\", \" \"]\n[15.667128, \"o\", \"\\u001b[4me\\u001b[24m\"]\n[15.794115, \"o\", \"\\b\\u001b[24med\"]\n[15.83337, \"o\", \"i\"]\n[16.711928, \"o\", \"t\"]\n[16.792473, \"o\", \" \"]\n[16.945941, \"o\", \"R\"]\n[17.022294, \"o\", \"E\"]\n[17.072731, \"o\", \"A\"]\n[17.247167, \"o\", \"D\"]\n[17.41645, \"o\", \"M\"]\n[17.546609, \"o\", \"E\"]\n[17.745329, \"o\", \".\"]\n[17.940659, \"o\", \"m\"]\n[18.027964, \"o\", \"d\"]\n[18.145754, \"o\", \"\\u001b[?1l\\u001b>\"]\n[18.151185, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[18.15313, \"o\", \"\\u001b]2;brig edit README.md\\u0007\\u001b]1;brig\\u0007\"]\n[18.311358, \"o\", \"\\u001b[?1000h\\u001b[?2004h\\u001b[?1049h\\u001b[22;0;0t\\u001b[?1h\\u001b=\\u001b[?2004h\"]\n[18.311704, \"o\", \"\\u001b[1;29r\\u001b[?12h\\u001b[?12l\\u001b[27m\\u001b[23m\\u001b[29m\\u001b[m\\u001b[H\\u001b[2J\\u001b[?25l\\u001b[29;1H\\\"/tmp/brig-cmd-buffer-7867430388356292876.md\\\"\"]\n[18.311802, \"o\", \" 15L, 443C\"]\n[18.327957, \"o\", \"\\u001b[2;1H▽\\u001b[6n\\u001b[2;1H \\u001b[1;1H\\u001b[>c\\u001b]10;?\\u0007\\u001b]11;?\\u0007\"]\n[18.330073, \"o\", \"\\u001b[1;1HWelcome to brig!\\r\\n\\r\\nHere's what you can do next:\\u001b[5;5H•\\u001b[5;7HAdd a few remotes to sync with (See 'brig remote add -h')\\r\\n •\\u001b[6;7HMount your data somewhere convinient (See 'brig mount -h')\\r\\n •\\u001b[7;7HHave a relaxing day exploring brig's features.\\r\\n\\r\\nPlease remember that brig is software in it's very early stages,\\r\\nand will currently eat your data with near-certainty.\\r\\n\\r\\nIf you're done with this README, you can easily remove it:\\u001b[14;5H$ brig rm README.md\\r\\n\\r\\n\\u001b[94m~ \\u001b[17;1H~ \\u001b[18;1H~ \\u001b[19;1H~ \\u001b[20;1H~ \"]\n[18.330257, \"o\", \" \\u001b[21;1H~ \\u001b[22;1H~ \\u001b[23;1H~ \\u001b[24;1H~ \\u001b[25;1H~ \\u001b[26;1H~ \\u001b[27;1H~ \\u001b[28;1H~ \"]\n[18.330339, \"o\", \" \\u001b[m\\u001b[29;102H1,1\\u001b[9CAlles\\u001b[1;1H\\u001b[?25h\"]\n[18.358891, \"o\", \"\\u001bP+q436f\\u001b\\\\\\u001bP+q6b75\\u001b\\\\\\u001bP+q6b64\\u001b\\\\\\u001bP+q6b72\\u001b\\\\\\u001bP+q6b6c\\u001b\\\\\\u001bP+q2332\\u001b\\\\\\u001bP+q2334\\u001b\\\\\\u001bP+q2569\\u001b\\\\\\u001bP+q2a37\\u001b\\\\\\u001bP+q6b31\\u001b\\\\\"]\n[18.374291, \"o\", \"\\u001b[?1000l\\u001b[?1006h\\u001b[?1002h\\u001b[?1006l\\u001b[?1002l\\u001b[?1006h\\u001b[?1002h\\u001b[27m\\u001b[23m\\u001b[29m\\u001b[m\\u001b[H\\u001b[2J\\u001b[?25l\\u001b[1;1HWelcome to brig!\\r\\n\\r\\nHere's what you can do next:\\u001b[5;5H•\\u001b[5;7HAdd a few remotes to sync with (See 'brig remote add -h')\\r\\n •\\u001b[6;7HMount your data somewhere convinient (See 'brig mount -h')\\r\\n •\\u001b[7;7HHave a relaxing day exploring brig's features.\\r\\n\\r\\nPlease remember that brig is software in it's very early stages,\\r\\nand will currently eat your data with near-certainty.\\r\\n\\r\\nIf you're done with this README, you can easily remove it:\\u001b[14;5H$ brig rm README.md\\r\\n\\r\\n\\u001b[94m~ \\u001b[17;1H~ \\u001b[18;1H~ \\u001b[19;1H~ \"]\n[18.374432, \"o\", \" \\u001b[20;1H~ \\u001b[21;1H~ \\u001b[22;1H~ \\u001b[23;1H~ \\u001b[24;1H~ \\u001b[25;1H~ \\u001b[26;1H~ \\u001b[27;1H~ \"]\n[18.374725, \"o\", \" \\u001b[28;1H~ \\u001b[m\\u001b[29;102H1,1\\u001b[9CAlles\\r\\\"/tmp/brig-cmd-buffer-7867430388356292876.md\\\" 15L, 443C\\u001b[1;1H\\u001b[?25h\"]\n[18.829942, \"o\", \"\\u001b[?25l\\u001b[29;92HG\\u001b[1;1H\"]\n[18.830099, \"o\", \"\\u001b[29;92H \\u001b[15;1H\\u001b[29;103H5,0-1\\u001b[15;1H\\u001b[?25h\"]\n[19.244409, \"o\", \"\\u001b[?25l\\u001b[29;92Ho\\u001b[15;1H\"]\n[19.244943, \"o\", \"\\u001b[29;92H \\u001b[16;1H\\u001b[29;1H\\u001b[1m-- EINFÜGEN --\\u001b[m\\u001b[29;15H\\u001b[K\\u001b[29;102H16,1\\u001b[8CAlles\"]\n[19.245973, \"o\", \"\\u001b[16;1H\\u001b[K\\u001b[16;1H\\u001b[?25h\"]\n[19.448015, \"o\", \"\\u001b[?25l\\u001b[17;1H\\u001b[K\\u001b[29;103H7\\u001b[17;1H\\u001b[?25h\"]\n[20.560184, \"o\", \"\\u001b[?25lA\\u001b[29;105H2\\u001b[17;2H\\u001b[?25h\"]\n[20.692311, \"o\", \"\\u001b[?25ln\\u001b[29;105H3\\u001b[17;3H\\u001b[?25h\"]\n[20.872813, \"o\", \"\\u001b[?25lo\\u001b[29;105H4\\u001b[17;4H\\u001b[?25h\"]\n[21.177969, \"o\", \"\\u001b[?25l\\u001b[17;3H\\u001b[K\\u001b[29;105H3\\u001b[17;3H\\u001b[?25h\"]\n[21.326674, \"o\", \"\\u001b[?25l\\u001b[17;2H\\u001b[K\\u001b[29;105H2\\u001b[17;2H\\u001b[?25h\"]\n[21.482987, \"o\", \"\\u001b[?25l\\u001b[17;1H\\u001b[K\\u001b[29;105H1\\u001b[17;1H\\u001b[?25h\"]\n[21.639899, \"o\", \"\\u001b[?25l\\u001b[94m~ \\u001b[m\\u001b[29;103H6\\u001b[16;1H\\u001b[?25h\"]\n[21.84207, \"o\", \"\\u001b[?25lO\\u001b[29;105H2\\u001b[16;2H\\u001b[?25h\"]\n[22.336138, \"o\", \"\\u001b[?25l\\u001b[16;1H\\u001b[K\\u001b[29;105H1\\u001b[16;1H\\u001b[?25h\"]\n[22.556694, \"o\", \"\\u001b[?25lO\\u001b[29;105H2\\u001b[16;2H\\u001b[?25h\"]\n[22.716563, \"o\", \"\\u001b[?25ln\\u001b[29;105H3\\u001b[16;3H\\u001b[?25h\"]\n[22.807098, \"o\", \"\\u001b[?25le\\u001b[29;105H4\\u001b[16;4H\\u001b[?25h\"]\n[22.853901, \"o\", \"\\u001b[?25l\\u001b[29;105H5\\u001b[16;5H\\u001b[?25h\"]\n[22.927145, \"o\", \"\\u001b[?25lm\\u001b[29;105H6\\u001b[16;6H\\u001b[?25h\"]\n[23.078599, \"o\", \"\\u001b[?25lo\\u001b[29;105H7\\u001b[16;7H\\u001b[?25h\"]\n[23.147662, \"o\", \"\\u001b[?25lr\\u001b[29;105H8\\u001b[16;8H\\u001b[?25h\"]\n[23.213428, \"o\", \"\\u001b[?25le\\u001b[29;105H9\\u001b[16;9H\\u001b[?25h\"]\n[23.237209, \"o\", \"\\u001b[?25l\\u001b[29;105H10\\u001b[16;10H\\u001b[?25h\"]\n[23.367679, \"o\", \"\\u001b[?25le\\u001b[29;106H1\\u001b[16;11H\\u001b[?25h\"]\n[23.506297, \"o\", \"\\u001b[?25ld\\u001b[29;106H2\\u001b[16;12H\\u001b[?25h\"]\n[23.561789, \"o\", \"\\u001b[?25li\\u001b[29;106H3\\u001b[16;13H\\u001b[?25h\"]\n[23.732131, \"o\", \"\\u001b[?25lt\\u001b[29;106H4\\u001b[16;14H\\u001b[?25h\"]\n[23.89776, \"o\", \"\\u001b[?25l.\\u001b[29;106H5\\u001b[16;15H\\u001b[?25h\"]\n[24.121988, \"o\", \"\\u001b[29;1H\\u001b[K\\u001b[16;14H\\u001b[?25l\\u001b[29;92H^[\\u001b[16;14H\"]\n[24.222361, \"o\", \"\\u001b[29;92H \\u001b[16;15H\"]\n[24.223161, \"o\", \"\\u001b[29;102H16,14\\u001b[7CAlles\\u001b[16;14H\\u001b[?25h\"]\n[24.480728, \"o\", \"\\u001b[?25l\\u001b[29;92H:\\u001b[16;14H\\u001b[29;92H\\u001b[K\\u001b[29;1H:\"]\n[24.480865, \"o\", \"\\u001b[?2004h\\u001b[?25h\"]\n[24.616449, \"o\", \"w\\u001b[?25l\\u001b[?25h\"]\n[24.680356, \"o\", \"q\\u001b[?25l\\u001b[?25h\"]\n[24.782547, \"o\", \"\\r\\u001b[?25l\"]\n[24.782777, \"o\", \"\\u001b[?1006l\\u001b[?1002l\\u001b[?2004l\\\"/tmp/brig-cmd-buffer-7867430388356292876.md\\\"\"]\n[24.783558, \"o\", \" 16L, 458C geschrieben\"]\n[24.785684, \"o\", \"\\r\\r\\r\\n\\u001b[?2004l\\u001b[?1l\\u001b>\\u001b[?25h\\u001b[?1049l\\u001b[23;0;0t\"]\n[24.976962, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[24.977439, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\"]\n[24.977573, \"o\", \"\\u001b]1;~/dev/brig/docs\\u0007\"]\n[24.977854, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[24.978174, \"o\", \"\\u001b[?1h\\u001b=\"]\n[24.97931, \"o\", \"\\u001b[?2004h\"]\n[26.39299, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[26.903388, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[27.043634, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[27.143328, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[27.220485, \"o\", \" \"]\n[27.432396, \"o\", \"\\u001b[4mm\\u001b[24m\"]\n[27.536401, \"o\", \"\\b\\u001b[24mmv\"]\n[27.604616, \"o\", \" \"]\n[28.387822, \"o\", \"s\"]\n[28.457765, \"o\", \"u\"]\n[28.599949, \"o\", \"b\"]\n[29.092484, \"o\", \"/\"]\n[29.516744, \"o\", \"m\"]\n[29.716171, \"o\", \"u\"]\n[29.805969, \"o\", \"s\"]\n[29.894391, \"o\", \"i\"]\n[29.986773, \"o\", \"c\"]\n[30.122947, \"o\", \".\"]\n[30.30213, \"o\", \"m\"]\n[30.46432, \"o\", \"p\"]\n[30.537035, \"o\", \"3\"]\n[30.6676, \"o\", \" \"]\n[31.453307, \"o\", \"s\"]\n[31.522437, \"o\", \"u\"]\n[31.647459, \"o\", \"b\"]\n[32.02632, \"o\", \"/\"]\n[32.528259, \"o\", \"e\"]\n[32.628939, \"o\", \"l\"]\n[32.781576, \"o\", \"s\"]\n[32.932367, \"o\", \"e\"]\n[33.02803, \"o\", \".\"]\n[33.198452, \"o\", \"m\"]\n[33.394547, \"o\", \"p\"]\n[33.498707, \"o\", \"3\"]\n[33.72191, \"o\", \"\\u001b[?1l\\u001b>\"]\n[33.728505, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[33.730485, \"o\", \"\\u001b]2;brig mv sub/music.mp3 sub/else.mp3\\u0007\\u001b]1;brig\\u0007\"]\n[33.852268, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[33.852394, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[33.852468, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[33.852541, \"o\", \"\\u001b[?1h\\u001b=\"]\n[33.852729, \"o\", \"\\u001b[?2004h\"]\n[34.937603, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[35.090575, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[35.180064, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[35.303067, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[35.386245, \"o\", \" \"]\n[35.509001, \"o\", \"d\"]\n[35.628174, \"o\", \"i\"]\n[35.711723, \"o\", \"f\"]\n[35.837246, \"o\", \"f\"]\n[36.037302, \"o\", \"\\u001b[?1l\\u001b>\"]\n[36.04126, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[36.043317, \"o\", \"\\u001b]2;brig diff\\u0007\\u001b]1;brig\\u0007\"]\n[36.175276, \"o\", \"\\u001b[35m•\\u001b[0m\\r\\n├──\\u001b[36m README.md ⇄ README.md\\u001b[0m\\r\\n└──sub\\r\\n └──\\u001b[34m else.mp3 → music.mp3\\u001b[0m\\r\\n\\r\\n0 directories, 3 files\\r\\n\"]\n[36.176742, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[36.176848, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\"]\n[36.176922, \"o\", \"\\u001b]1;~/dev/brig/docs\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[36.177004, \"o\", \"\\u001b[?1h\\u001b=\"]\n[36.177226, \"o\", \"\\u001b[?2004h\"]\n[39.463756, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n" }, { "path": "docs/asciinema/6_history.json", "content": "{\"version\": 2, \"width\": 119, \"height\": 29, \"timestamp\": 1519815271, \"env\": {\"SHELL\": \"/bin/zsh\", \"TERM\": \"xterm-256color\"}}\n[0.247164, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[0.247332, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[0.2474, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[0.247473, \"o\", \"\\u001b[?1h\\u001b=\"]\n[0.247687, \"o\", \"\\u001b[?2004h\"]\n[0.758319, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[0.903542, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[0.991215, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[1.097336, \"o\", \"\\b\\u001b[1m\\u001b[31mi\\u001b[1m\\u001b[31mh\\u001b[0m\\u001b[39m\"]\n[1.234071, \"o\", \" \"]\n[1.604765, \"o\", \"\\b\"]\n[1.742115, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\\u001b[0m\\u001b[39m \\b\"]\n[1.819036, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[1.874837, \"o\", \" \"]\n[2.066405, \"o\", \"h\"]\n[2.230528, \"o\", \"i\"]\n[2.308608, \"o\", \"s\"]\n[2.441739, \"o\", \"t\"]\n[2.556275, \"o\", \"o\"]\n[2.631017, \"o\", \"r\"]\n[2.782399, \"o\", \"y\"]\n[2.849719, \"o\", \" \"]\n[2.992586, \"o\", \"n\"]\n[3.076498, \"o\", \"e\"]\n[3.244405, \"o\", \"w\"]\n[3.343333, \"o\", \"-\"]\n[3.443824, \"o\", \"f\"]\n[3.517283, \"o\", \"i\"]\n[3.661094, \"o\", \"l\"]\n[3.760592, \"o\", \"e\"]\n[3.867321, \"o\", \"\\u001b[?1l\\u001b>\"]\n[3.873454, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[3.875831, \"o\", \"\\u001b]2;brig history new-file\\u0007\\u001b]1;brig\\u0007\"]\n[3.994567, \"o\", \"CHANGE FROM TO WHEN \\r\\n\\u001b[33madded\\u001b[0m\"]\n[3.994631, \"o\", \" \\u001b[36mSEfXUCU47p\\u001b[0m \\u001b[32mHEAD\\u001b[0m \\u001b[35mFeb 28 10:45:21\\u001b[0m \\r\\n\"]\n[3.996106, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[3.996207, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[3.996274, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[3.996336, \"o\", \"\\u001b[?1h\\u001b=\"]\n[3.996539, \"o\", \"\\u001b[?2004h\"]\n[4.612566, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[4.750317, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[4.85455, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[4.948061, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[5.035556, \"o\", \" \"]\n[5.136146, \"o\", \"\\u001b[4me\\u001b[24m\"]\n[5.266322, \"o\", \"\\b\\u001b[24med\"]\n[5.368632, \"o\", \"i\"]\n[5.518402, \"o\", \"t\"]\n[5.592434, \"o\", \" \"]\n[5.747638, \"o\", \"n\"]\n[5.811149, \"o\", \"e\"]\n[5.988417, \"o\", \"w\"]\n[6.102252, \"o\", \"-\"]\n[6.216878, \"o\", \"f\"]\n[6.308529, \"o\", \"i\"]\n[6.462365, \"o\", \"l\"]\n[6.568562, \"o\", \"e\"]\n[6.966448, \"o\", \"\\u001b[?1l\\u001b>\"]\n[6.972753, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[6.974692, \"o\", \"\\u001b]2;brig edit new-file\\u0007\\u001b]1;brig\\u0007\"]\n[7.098775, \"o\", \"\\u001b[?1000h\\u001b[?2004h\\u001b[?1049h\\u001b[22;0;0t\\u001b[?1h\\u001b=\\u001b[?2004h\"]\n[7.099102, \"o\", \"\\u001b[1;29r\\u001b[?12h\\u001b[?12l\\u001b[27m\\u001b[23m\\u001b[29m\\u001b[m\\u001b[H\\u001b[2J\\u001b[?25l\\u001b[29;1H\\\"/tmp/brig-cmd-buffer-1040861570544051988\\\"\"]\n[7.099183, \"o\", \" 1L, 72C\"]\n[7.102658, \"o\", \"\\u001b[2;1H▽\\u001b[6n\\u001b[2;1H \\u001b[1;1H\\u001b[>c\\u001b]10;?\\u0007\\u001b]11;?\\u0007\"]\n[7.102931, \"o\", \"\\u001b[1;1HThis is a new file and I can use this folder just like a normal folder.\\r\\n\\u001b[94m~ \\u001b[3;1H~ \\u001b[4;1H~ \\u001b[5;1H~ \\u001b[6;1H~ \\u001b[7;1H~ \\u001b[8;1H~ \\u001b[9;1H~ \"]\n[7.103048, \"o\", \" \\u001b[10;1H~ \\u001b[11;1H~ \\u001b[12;1H~ \\u001b[13;1H~ \\u001b[14;1H~ \\u001b[15;1H~ \\u001b[16;1H~ \\u001b[17;1H~ \"]\n[7.103117, \"o\", \" \\u001b[18;1H~ \\u001b[19;1H~ \\u001b[20;1H~ \\u001b[21;1H~ \\u001b[22;1H~ \\u001b[23;1H~ \\u001b[24;1H~ \\u001b[25;1H~ \"]\n[7.103178, \"o\", \" \\u001b[26;1H~ \\u001b[27;1H~ \\u001b[28;1H~ \\u001b[m\\u001b[29;102H1,1\\u001b[9CAlles\\u001b[1;1H\\u001b[?25h\"]\n[7.115208, \"o\", \"\\u001bP+q436f\\u001b\\\\\\u001bP+q6b75\\u001b\\\\\\u001bP+q6b64\\u001b\\\\\\u001bP+q6b72\\u001b\\\\\\u001bP+q6b6c\\u001b\\\\\\u001bP+q2332\\u001b\\\\\\u001bP+q2334\\u001b\\\\\\u001bP+q2569\\u001b\\\\\\u001bP+q2a37\\u001b\\\\\\u001bP+q6b31\\u001b\\\\\"]\n[7.118422, \"o\", \"\\u001b[?1000l\\u001b[?1006h\\u001b[?1002h\\u001b[?1006l\\u001b[?1002l\\u001b[?1006h\\u001b[?1002h\\u001b[27m\\u001b[23m\\u001b[29m\\u001b[m\\u001b[H\\u001b[2J\\u001b[?25l\\u001b[1;1HThis is a new file and I can use this folder just like a normal folder.\\r\\n\\u001b[94m~ \\u001b[3;1H~ \\u001b[4;1H~ \\u001b[5;1H~ \\u001b[6;1H~ \\u001b[7;1H~ \\u001b[8;1H~ \"]\n[7.119068, \"o\", \" \\u001b[9;1H~ \\u001b[10;1H~ \\u001b[11;1H~ \\u001b[12;1H~ \\u001b[13;1H~ \\u001b[14;1H~ \\u001b[15;1H~ \\u001b[16;1H~ \\u001b\"]\n[7.119231, \"o\", \"[17;1H~ \\u001b[18;1H~ \\u001b[19;1H~ \\u001b[20;1H~ \\u001b[21;1H~ \\u001b[22;1H~ \\u001b[23;1H~ \\u001b[24;1H~ \\u001b[25;1H~ \"]\n[7.11934, \"o\", \" \\u001b[26;1H~ \\u001b[27;1H~ \\u001b[28;1H~ \\u001b[m\\u001b[29;102H1,1\\u001b[9CAlles\\r\\\"/tmp/brig-cmd-buffer-1040861570544051988\\\" 1L, 72C\\u001b[1;1H\\u001b[?25h\"]\n[8.390161, \"o\", \"\\u001b[?25l\\u001b[29;92Ho\\u001b[1;1H\\u001b[29;92H \\u001b[2;1H\\u001b[29;1H\\u001b[1m-- EINFÜGEN --\\u001b[m\\u001b[29;15H\\u001b[K\\u001b[29;102H2,1\\u001b[9CAlles\"]\n[8.390573, \"o\", \"\\u001b[2;1H\\u001b[K\\u001b[2;1H\\u001b[?25h\"]\n[8.7854, \"o\", \"\\u001b[?25lI\\u001b[29;104H2\\u001b[2;2H\\u001b[?25h\"]\n[8.936335, \"o\", \"\\u001b[?25l\\u001b[29;104H3\\u001b[2;3H\\u001b[?25h\"]\n[9.210809, \"o\", \"\\u001b[?25la\\u001b[29;104H4\\u001b[2;4H\\u001b[?25h\"]\n[9.316575, \"o\", \"\\u001b[?25ld\\u001b[29;104H5\\u001b[2;5H\\u001b[?25h\"]\n[9.485026, \"o\", \"\\u001b[?25ld\\u001b[29;104H6\\u001b[2;6H\\u001b[?25h\"]\n[9.589198, \"o\", \"\\u001b[?25le\\u001b[29;104H7\\u001b[2;7H\\u001b[?25h\"]\n[9.68663, \"o\", \"\\u001b[?25ld\\u001b[29;104H8\\u001b[2;8H\\u001b[?25h\"]\n[9.762494, \"o\", \"\\u001b[?25l\\u001b[29;104H9\\u001b[2;9H\\u001b[?25h\"]\n[9.928953, \"o\", \"\\u001b[?25la\\u001b[29;104H10\\u001b[2;10H\\u001b[?25h\"]\n[10.009957, \"o\", \"\\u001b[?25ln\\u001b[29;105H1\\u001b[2;11H\\u001b[?25h\"]\n[10.181435, \"o\", \"\\u001b[?25lo\\u001b[29;105H2\\u001b[2;12H\\u001b[?25h\"]\n[10.277076, \"o\", \"\\u001b[?25lt\\u001b[29;105H3\\u001b[2;13H\\u001b[?25h\"]\n[10.380755, \"o\", \"\\u001b[?25lh\\u001b[29;105H4\\u001b[2;14H\\u001b[?25h\"]\n[10.467393, \"o\", \"\\u001b[?25le\\u001b[29;105H5\\u001b[2;15H\\u001b[?25h\"]\n[10.515522, \"o\", \"\\u001b[?25lr\\u001b[29;105H6\\u001b[2;16H\\u001b[?25h\"]\n[10.599308, \"o\", \"\\u001b[?25l\\u001b[29;105H7\\u001b[2;17H\\u001b[?25h\"]\n[10.678216, \"o\", \"\\u001b[?25ll\\u001b[29;105H8\\u001b[2;18H\\u001b[?25h\"]\n[10.812556, \"o\", \"\\u001b[?25li\\u001b[29;105H9\\u001b[2;19H\\u001b[?25h\"]\n[10.972619, \"o\", \"\\u001b[?25ln\\u001b[29;104H20\\u001b[2;20H\\u001b[?25h\"]\n[11.036455, \"o\", \"\\u001b[?25le\\u001b[29;105H1\\u001b[2;21H\\u001b[?25h\"]\n[11.140062, \"o\", \"\\u001b[?25l\\u001b[29;105H2\\u001b[2;22H\\u001b[?25h\"]\n[11.219498, \"o\", \"\\u001b[?25lt\\u001b[29;105H3\\u001b[2;23H\\u001b[?25h\"]\n[11.330255, \"o\", \"\\u001b[?25lo\\u001b[29;105H4\\u001b[2;24H\\u001b[?25h\"]\n[11.394367, \"o\", \"\\u001b[?25l\\u001b[29;105H5\\u001b[2;25H\\u001b[?25h\"]\n[11.487906, \"o\", \"\\u001b[?25ls\\u001b[29;105H6\\u001b[2;26H\\u001b[?25h\"]\n[11.550727, \"o\", \"\\u001b[?25lh\\u001b[29;105H7\\u001b[2;27H\\u001b[?25h\"]\n[11.714439, \"o\", \"\\u001b[?25lo\\u001b[29;105H8\\u001b[2;28H\\u001b[?25h\"]\n[11.82308, \"o\", \"\\u001b[?25lw\\u001b[29;105H9\\u001b[2;29H\\u001b[?25h\"]\n[11.911686, \"o\", \"\\u001b[?25l\\u001b[29;104H30\\u001b[2;30H\\u001b[?25h\"]\n[12.008968, \"o\", \"\\u001b[?25lt\\u001b[29;105H1\\u001b[2;31H\\u001b[?25h\"]\n[12.089608, \"o\", \"\\u001b[?25lh\\u001b[29;105H2\\u001b[2;32H\\u001b[?25h\"]\n[12.179902, \"o\", \"\\u001b[?25le\\u001b[29;105H3\\u001b[2;33H\\u001b[?25h\"]\n[12.216415, \"o\", \"\\u001b[?25l\\u001b[29;105H4\\u001b[2;34H\\u001b[?25h\"]\n[12.425471, \"o\", \"\\u001b[?25lh\\u001b[29;105H5\\u001b[2;35H\\u001b[?25h\"]\n[12.555247, \"o\", \"\\u001b[?25li\\u001b[29;105H6\\u001b[2;36H\\u001b[?25h\"]\n[12.661093, \"o\", \"\\u001b[?25ls\\u001b[29;105H7\\u001b[2;37H\\u001b[?25h\"]\n[12.798374, \"o\", \"\\u001b[?25ly\\u001b[29;105H8\\u001b[2;38H\\u001b[?25h\"]\n[12.891934, \"o\", \"\\u001b[?25lo\\u001b[29;105H9\\u001b[2;39H\\u001b[?25h\"]\n[12.999356, \"o\", \"\\u001b[?25lr\\u001b[29;104H40\\u001b[2;40H\\u001b[?25h\"]\n[13.307156, \"o\", \"\\u001b[?25l\\u001b[2;39H\\u001b[K\\u001b[29;104H39\\u001b[2;39H\\u001b[?25h\"]\n[13.446672, \"o\", \"\\u001b[?25l\\u001b[2;38H\\u001b[K\\u001b[29;105H8\\u001b[2;38H\\u001b[?25h\"]\n[13.583738, \"o\", \"\\u001b[?25l\\u001b[2;37H\\u001b[K\\u001b[29;105H7\\u001b[2;37H\\u001b[?25h\"]\n[13.680682, \"o\", \"\\u001b[?25lt\\u001b[29;105H8\\u001b[2;38H\\u001b[?25h\"]\n[13.774376, \"o\", \"\\u001b[?25lo\\u001b[29;105H9\\u001b[2;39H\\u001b[?25h\"]\n[13.861644, \"o\", \"\\u001b[?25lr\\u001b[29;104H40\\u001b[2;40H\\u001b[?25h\"]\n[14.015369, \"o\", \"\\u001b[?25ly\\u001b[29;105H1\\u001b[2;41H\\u001b[?25h\"]\n[14.086862, \"o\", \"\\u001b[?25l\\u001b[29;105H2\\u001b[2;42H\\u001b[?25h\"]\n[14.235689, \"o\", \"\\u001b[?25lf\\u001b[29;105H3\\u001b[2;43H\\u001b[?25h\"]\n[14.935863, \"o\", \"\\u001b[?25le\\u001b[29;105H4\\u001b[2;44H\\u001b[?25h\"]\n[15.011571, \"o\", \"\\u001b[?25la\\u001b[29;105H5\\u001b[2;45H\\u001b[?25h\"]\n[15.144528, \"o\", \"\\u001b[?25lt\\u001b[29;105H6\\u001b[2;46H\\u001b[?25h\"]\n[15.26693, \"o\", \"\\u001b[?25lu\\u001b[29;105H7\\u001b[2;47H\\u001b[?25h\"]\n[15.337705, \"o\", \"\\u001b[?25lr\\u001b[29;105H8\\u001b[2;48H\\u001b[?25h\"]\n[15.408419, \"o\", \"\\u001b[?25le\\u001b[29;105H9\\u001b[2;49H\\u001b[?25h\"]\n[15.520857, \"o\", \"\\u001b[?25l.\\u001b[29;104H50\\u001b[2;50H\\u001b[?25h\"]\n[15.684168, \"o\", \"\\u001b[29;1H\\u001b[K\\u001b[2;49H\\u001b[?25l\\u001b[29;92H^[\\u001b[2;49H\"]\n[15.784483, \"o\", \"\\u001b[29;92H \\u001b[2;50H\"]\n[15.785392, \"o\", \"\\u001b[29;102H2,49\\u001b[8CAlles\\u001b[2;49H\\u001b[?25h\"]\n[15.965731, \"o\", \"\\u001b[?25l\\u001b[29;92H:\\u001b[2;49H\"]\n[15.965848, \"o\", \"\\u001b[29;92H\\u001b[K\\u001b[29;1H:\\u001b[?2004h\\u001b[?25h\"]\n[16.0631, \"o\", \"w\\u001b[?25l\\u001b[?25h\"]\n[16.130332, \"o\", \"q\\u001b[?25l\\u001b[?25h\"]\n[16.237935, \"o\", \"\\r\"]\n[16.238532, \"o\", \"\\u001b[?25l\\u001b[?1006l\\u001b[?1002l\\u001b[?2004l\\\"/tmp/brig-cmd-buffer-1040861570544051988\\\"\"]\n[16.239785, \"o\", \" 2L, 122C geschrieben\"]\n[16.242119, \"o\", \"\\r\\r\\r\\n\\u001b[?2004l\\u001b[?1l\\u001b>\\u001b[?25h\\u001b[?1049l\\u001b[23;0;0t\"]\n[16.483625, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[16.483851, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \"]\n[16.484182, \"o\", \"\\u001b[K\\u001b[?1h\\u001b=\"]\n[16.484373, \"o\", \"\\u001b[?2004h\"]\n[16.649917, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[16.74022, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[16.863539, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[16.933108, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[17.02806, \"o\", \" \"]\n[17.139172, \"o\", \"\\u001b[4mc\\u001b[24m\"]\n[17.196635, \"o\", \"\\b\\u001b[4mc\\u001b[4mo\\u001b[24m\"]\n[17.369386, \"o\", \"\\b\\u001b[4mo\\u001b[4mm\\u001b[24m\"]\n[17.506413, \"o\", \"\\b\\b\\b\\u001b[24mc\\u001b[24mo\\u001b[24mmm\"]\n[17.70377, \"o\", \"i\"]\n[17.781604, \"o\", \"t\"]\n[17.844497, \"o\", \" \"]\n[17.932295, \"o\", \"-\"]\n[18.147048, \"o\", \"m\"]\n[19.411942, \"o\", \" \"]\n[19.579929, \"o\", \"\\u001b[33m'\\u001b[39m\"]\n[20.289417, \"o\", \"\\b\\u001b[33m'\\u001b[33me\\u001b[39m\"]\n[20.429284, \"o\", \"\\b\\u001b[33me\\u001b[33md\\u001b[39m\"]\n[20.564702, \"o\", \"\\b\\u001b[33md\\u001b[33mi\\u001b[39m\"]\n[20.64266, \"o\", \"\\b\\u001b[33mi\\u001b[33mt\\u001b[39m\"]\n[20.694368, \"o\", \"\\b\\u001b[33mt\\u001b[33me\\u001b[39m\"]\n[20.818051, \"o\", \"\\b\\u001b[33me\\u001b[33md\\u001b[39m\"]\n[20.908198, \"o\", \"\\b\\u001b[33md\\u001b[33m \\u001b[39m\"]\n[20.970043, \"o\", \"\\b\\u001b[33m \\u001b[33mn\\u001b[39m\"]\n[21.013369, \"o\", \"\\b\\u001b[33mn\\u001b[33me\\u001b[39m\"]\n[21.171105, \"o\", \"\\b\\u001b[33me\\u001b[33mw\\u001b[39m\"]\n[21.240077, \"o\", \"\\b\\u001b[33mw\\u001b[33m-\\u001b[39m\"]\n[21.33241, \"o\", \"\\b\\u001b[33m-\\u001b[33mf\\u001b[39m\"]\n[21.421937, \"o\", \"\\b\\u001b[33mf\\u001b[33mi\\u001b[39m\"]\n[21.581226, \"o\", \"\\b\\u001b[33mi\\u001b[33ml\\u001b[39m\"]\n[21.694969, \"o\", \"\\b\\u001b[33ml\\u001b[33me\\u001b[39m\"]\n[21.839741, \"o\", \"\\b\\u001b[33me\\u001b[33m'\\u001b[39m\"]\n[22.007146, \"o\", \"\\u001b[?1l\\u001b>\"]\n[22.013129, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[22.01528, \"o\", \"\\u001b]2;brig commit -m 'edited new-file'\\u0007\\u001b]1;brig\\u0007\"]\n[22.139023, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[22.139152, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[22.13918, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[22.139295, \"o\", \"\\u001b[?1h\\u001b=\"]\n[22.13949, \"o\", \"\\u001b[?2004h\"]\n[23.020604, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[23.13374, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[23.26246, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[23.335487, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[23.437349, \"o\", \" \"]\n[23.917417, \"o\", \"\\u001b[4ml\\u001b[24m\"]\n[24.065659, \"o\", \"\\b\\u001b[4ml\\u001b[4mo\\u001b[24m\"]\n[24.14673, \"o\", \"\\b\\u001b[4mo\\u001b[4mg\\u001b[24m\"]\n[24.248101, \"o\", \"\\u001b[?1l\\u001b>\"]\n[24.253336, \"o\", \"\\u001b[?2004l\\b\\b\\b\\u001b[24ml\\u001b[24mo\\u001b[24mg\\r\\r\\n\"]\n[24.255808, \"o\", \"\\u001b]2;brig log\\u0007\\u001b]1;brig\\u0007\"]\n[24.368164, \"o\", \"\\u001b[32mSEfXUERkEf\\u001b[0m \\u001b[33mFeb 28 11:54:53\\u001b[0m \\u001b[31m•\\u001b[0m\\u001b[36m (curr)\\u001b[0m\\r\\n\"]\n[24.368292, \"o\", \"\\u001b[32mSEfXUBmLmQ\\u001b[0m \\u001b[33mFeb 28 10:59:36\\u001b[0m edited new-file\\u001b[36m (head)\\u001b[0m\\r\\n\\u001b[32mSEfXUCjYxA\\u001b[0m \\u001b[33mFeb 28 10:45:21\\u001b[0m Added darth vader music\\u001b[36m\\u001b[0m\\r\\n\\u001b[32mSEfXUCU47p\\u001b[0m \\u001b[33mFeb 28 10:45:21\\u001b[0m Added initial README.md\\u001b[36m\\u001b[0m\\r\\n\"]\n[24.368379, \"o\", \"\\u001b[32mSEfXUCEaXL\\u001b[0m \\u001b[33mFeb 28 10:45:21\\u001b[0m initial commit\\u001b[36m (init)\\u001b[0m\\r\\n\"]\n[24.369869, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[24.370075, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[24.370148, \"o\", \"\\u001b[?1h\\u001b=\"]\n[24.370345, \"o\", \"\\u001b[?2004h\"]\n[26.14913, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[26.246472, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[26.346298, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[26.437241, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[30.638228, \"o\", \" \"]\n[30.201934, \"o\", \"h\"]\n[30.329335, \"o\", \"i\"]\n[30.411966, \"o\", \"s\"]\n[30.557041, \"o\", \"y\"]\n[30.647071, \"o\", \"o\"]\n[30.764317, \"o\", \"r\"]\n[31.0818, \"o\", \"\\b \\b\"]\n[31.228267, \"o\", \"\\b \\b\"]\n[31.376425, \"o\", \"\\b \\b\"]\n[31.504246, \"o\", \"t\"]\n[31.605135, \"o\", \"o\"]\n[31.697515, \"o\", \"r\"]\n[31.868937, \"o\", \"y\"]\n[31.949671, \"o\", \" \"]\n[32.481463, \"o\", \"n\"]\n[32.540945, \"o\", \"e\"]\n[32.706198, \"o\", \"w\"]\n[32.780756, \"o\", \"-\"]\n[32.893561, \"o\", \"f\"]\n[32.984879, \"o\", \"i\"]\n[33.131834, \"o\", \"l\"]\n[33.235133, \"o\", \"e\"]\n[33.29174, \"o\", \"\\u001b[?1l\\u001b>\"]\n[33.298203, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[33.30009, \"o\", \"\\u001b]2;brig history new-file\\u0007\\u001b]1;brig\\u0007\"]\n[33.411816, \"o\", \"CHANGE FROM TO WHEN \\r\\n\\u001b[33mmodified\\u001b[0m \\u001b[36mSEfXUCjYxA\\u001b[0m \\u001b[32mHEAD\\u001b[0m \\u001b[35mFeb 28 10:59:36\\u001b[0m \\r\\n\\u001b[33madded\\u001b[0m\"]\n[33.411883, \"o\", \" \\u001b[36mSEfXUCU47p\\u001b[0m \\u001b[32mSEfXUCjYxA\\u001b[0m \\u001b[35mFeb 28 10:45:21\\u001b[0m \\r\\n\"]\n[33.413372, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[33.413489, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[33.413564, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[33.413654, \"o\", \"\\u001b[?1h\\u001b=\"]\n[33.413868, \"o\", \"\\u001b[?2004h\"]\n[34.337473, \"o\", \"\\u001b[32mbrig\\u001b[39m history new-file\"]\n[34.823467, \"o\", \"\\b\\b\\b\\b \\b\\b\\b\\b\"]\n[35.02909, \"o\", \"\\b\\b\\b\\b \\b\\b\\b\\b\"]\n[35.651468, \"o\", \"\\u001b[8D \\b\\b\\b\\b\\b\\b\\b\"]\n[35.930863, \"o\", \"\\u001b[4mr\\u001b[24m\"]\n[35.992925, \"o\", \"\\b\\u001b[4mr\\u001b[4me\\u001b[24m\"]\n[36.177469, \"o\", \"\\b\\b\\u001b[24mr\\u001b[24mes\"]\n[36.343523, \"o\", \"e\"]\n[36.425736, \"o\", \"t\"]\n[36.475568, \"o\", \" \"]\n[37.198725, \"o\", \"n\"]\n[37.297678, \"o\", \"e\"]\n[37.66047, \"o\", \"\\b \\b\"]\n[37.811151, \"o\", \"\\b \\b\"]\n[38.581713, \"o\", \"H\"]\n[38.674511, \"o\", \"E\"]\n[38.720165, \"o\", \"A\"]\n[38.849624, \"o\", \"D\"]\n[39.521639, \"o\", \"^\"]\n[39.629115, \"o\", \" \"]\n[39.97663, \"o\", \"n\"]\n[40.071728, \"o\", \"e\"]\n[40.241242, \"o\", \"w\"]\n[40.336104, \"o\", \"-\"]\n[40.44449, \"o\", \"f\"]\n[40.546932, \"o\", \"i\"]\n[40.701172, \"o\", \"l\"]\n[40.796942, \"o\", \"e\"]\n[40.88402, \"o\", \"\\u001b[?1l\\u001b>\"]\n[40.890725, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[40.892491, \"o\", \"\\u001b]2;brig reset HEAD^ new-file\\u0007\\u001b]1;brig\\u0007\"]\n[41.011663, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[41.011779, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[41.011852, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[41.011931, \"o\", \"\\u001b[?1h\\u001b=\"]\n[41.012152, \"o\", \"\\u001b[?2004h\"]\n[43.678464, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[43.771458, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[43.869942, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[43.974751, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[44.052256, \"o\", \" \"]\n[44.319182, \"o\", \"\\u001b[4mc\\u001b[24m\"]\n[44.389519, \"o\", \"\\b\\u001b[24mca\"]\n[44.537355, \"o\", \"t\"]\n[44.605275, \"o\", \" \"]\n[45.401779, \"o\", \"n\"]\n[45.476219, \"o\", \"e\"]\n[45.63769, \"o\", \"w\"]\n[45.742511, \"o\", \"-\"]\n[45.841412, \"o\", \"f\"]\n[45.935062, \"o\", \"i\"]\n[46.094286, \"o\", \"l\"]\n[46.199279, \"o\", \"e\"]\n[46.29546, \"o\", \"\\u001b[?1l\\u001b>\"]\n[46.301738, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[46.303773, \"o\", \"\\u001b]2;brig cat new-file\\u0007\\u001b]1;brig\\u0007\"]\n[46.462339, \"o\", \"This is a new file and I can use this folder just like a normal folder.\\r\\n\"]\n[46.465781, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[46.465999, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[46.466058, \"o\", \"\\u001b[?1h\\u001b=\"]\n[46.466267, \"o\", \"\\u001b[?2004h\"]\n[50.068751, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n" }, { "path": "docs/asciinema/7_remotes.json", "content": "{\"version\": 2, \"width\": 119, \"height\": 29, \"timestamp\": 1519833254, \"env\": {\"SHELL\": \"/bin/zsh\", \"TERM\": \"xterm-256color\"}, \"title\": \"brig remote\"}\n[0.256979, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[0.257127, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[0.257195, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[0.257294, \"o\", \"\\u001b[?1h\\u001b=\"]\n[0.257515, \"o\", \"\\u001b[?2004h\"]\n[0.452491, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[0.592882, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[0.657491, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[0.768666, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[0.836203, \"o\", \" \"]\n[0.935009, \"o\", \"w\"]\n[1.089043, \"o\", \"h\"]\n[1.16401, \"o\", \"o\"]\n[1.309524, \"o\", \"a\"]\n[1.374321, \"o\", \"m\"]\n[1.554943, \"o\", \"i\"]\n[1.864501, \"o\", \"\\u001b[?1l\\u001b>\"]\n[1.868891, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[1.871071, \"o\", \"\\u001b]2;brig whoami\\u0007\\u001b]1;brig\\u0007\"]\n[2.029988, \"o\", \"- Name: \\u001b[33msahib@wald.de/laptop\\u001b[0m\\r\\n Fingerprint: QmXdjMyoLSYzE5r2v1LFZFkgDtNjtcLZZuoHesg3temwLH:SEfXUDYVaZuHryzkXmQvsD35qYz9iBZwnpnRVQvwJjB5yWu2Ygqzbu23CCWUyvk7GVtmwg5m5j93b4S9QNhNs7UXmkPid\\r\\n\"]\n[2.031508, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[2.031622, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[2.031701, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[2.031773, \"o\", \"\\u001b[?1h\\u001b=\"]\n[2.03198, \"o\", \"\\u001b[?2004h\"]\n[3.57628, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[3.690324, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[3.806995, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[3.880188, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[3.961119, \"o\", \" \"]\n[4.064671, \"o\", \"n\"]\n[4.144889, \"o\", \"e\"]\n[4.249377, \"o\", \"t\"]\n[4.299546, \"o\", \" \"]\n[4.433507, \"o\", \"\\u001b[4ml\\u001b[24m\"]\n[4.59375, \"o\", \"\\b\\u001b[4ml\\u001b[4mo\\u001b[24m\"]\n[4.750709, \"o\", \"\\b\\b\\u001b[24ml\\u001b[24moc\"]\n[4.794775, \"o\", \"a\"]\n[4.943057, \"o\", \"t\"]\n[5.005801, \"o\", \"e\"]\n[5.036716, \"o\", \" \"]\n[5.521982, \"o\", \"\\u001b[4ma\\u001b[24m\"]\n[5.66573, \"o\", \"\\b\\u001b[24mal\"]\n[5.849781, \"o\", \"i\"]\n[5.949514, \"o\", \"c\"]\n[5.993024, \"o\", \"e\"]\n[6.615229, \"o\", \"\\u001b[?1l\\u001b>\"]\n[6.624865, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[6.627092, \"o\", \"\\u001b]2;brig net locate alice\\u0007\\u001b]1;brig\\u0007\"]\n[7.24608, \"o\", \"Scanning. \\r\"]\n[7.746166, \"o\", \"Scanning.. \\r\"]\n[8.246136, \"o\", \"Scanning... \\r\"]\n[8.746117, \"o\", \"Scanning.... \\r\"]\n[9.246041, \"o\", \"Scanning.....\\r\"]\n[9.746112, \"o\", \"Scanning. \\r\"]\n[10.246042, \"o\", \"Scanning.. \\r\"]\n[10.746183, \"o\", \"Scanning... \\r\"]\n[11.246097, \"o\", \"Scanning.... \\r\"]\n[11.746129, \"o\", \"Scanning.....\\r\"]\n[12.246214, \"o\", \"Scanning. \\r\"]\n[12.746018, \"o\", \"Scanning.. \\r\"]\n[13.246042, \"o\", \"Scanning... \\r\"]\n[13.746093, \"o\", \"Scanning.... \\r\"]\n[14.246131, \"o\", \"Scanning.....\\r\"]\n[14.746065, \"o\", \"Scanning. \\r\"]\n[15.246104, \"o\", \"Scanning.. \\r\"]\n[15.746047, \"o\", \"Scanning... \\r\"]\n[16.246048, \"o\", \"Scanning.... \\r\"]\n[16.746052, \"o\", \"Scanning.....\\r\"]\n[16.776321, \"o\", \"NAME TYPE FINGERPRINT\\r\\nalice@wonderland.lit/container email \\u001b[32mQmUzLSHCKUNDow8YS6tEZAPzKYVqoG8oYj41ioN1YgHWCb:SEfXUDA157kQoqRc6EvwRy3v3kFdTJh2HXcWso7HBGJVMi7WhCC476Gp6BHMQKtZiVuJqc4o43RZ1dz3dPeJEgLyMcRTB\\u001b[0m\\r\\n\"]\n[18.54981, \"o\", \"^C\"]\n[18.555759, \"o\", \"\\r\\n\"]\n[18.556274, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[18.556666, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\"]\n[18.557775, \"o\", \"\\u001b]1;~/dev/brig/docs\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\\u001b[?1h\\u001b=\"]\n[18.558285, \"o\", \"\\u001b[?2004h\"]\n[19.449127, \"o\", \"\\u001b[1m\\u001b[31mn\\u001b[0m\\u001b[39m\"]\n[19.548774, \"o\", \"\\b\\u001b[1m\\u001b[31mn\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[19.644313, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mn\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[19.732335, \"o\", \"\\b\\u001b[1m\\u001b[31mi\\u001b[1m\\u001b[31mg\\u001b[0m\\u001b[39m\"]\n[19.796273, \"o\", \" \"]\n[20.355635, \"o\", \"\\b\\b\\b\\b\\b\\u001b[0m\\u001b[39m \\u001b[0m\\u001b[39m \\u001b[0m\\u001b[39m \\u001b[0m\\u001b[39m \\b\\b\\b\\b\"]\n[20.610364, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[20.713316, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[20.842937, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[20.943321, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[21.056141, \"o\", \" \"]\n[21.20604, \"o\", \"\\u001b[4mr\\u001b[24m\"]\n[21.269933, \"o\", \"\\b\\u001b[4mr\\u001b[4me\\u001b[24m\"]\n[21.303008, \"o\", \"\\b\\b\\u001b[24mr\\u001b[24mem\"]\n[21.466296, \"o\", \"o\"]\n[21.515036, \"o\", \"t\"]\n[21.558255, \"o\", \"e\"]\n[21.595261, \"o\", \" \"]\n[21.701607, \"o\", \"\\u001b[4ml\\u001b[24m\"]\n[21.790604, \"o\", \"\\b\\u001b[24mls\"]\n[21.875449, \"o\", \"\\u001b[?1l\\u001b>\"]\n[21.880974, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[21.883036, \"o\", \"\\u001b]2;brig remote ls\\u0007\\u001b]1;brig\\u0007\"]\n[22.033551, \"o\", \"None yet. Use `brig remote add ` to add some.\\r\\n\"]\n[22.034945, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[22.035119, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\"]\n[22.03519, \"o\", \"\\u001b]1;~/dev/brig/docs\\u0007\"]\n[22.035295, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[22.035476, \"o\", \"\\u001b[?1h\\u001b=\"]\n[22.035744, \"o\", \"\\u001b[?2004h\"]\n[23.350038, \"o\", \"\\u001b[32mbrig\\u001b[39m remote ls\"]\n[23.797034, \"o\", \"\\b\\b \\b\\b\"]\n[24.100147, \"o\", \"\\u001b[4ma\\u001b[24m\"]\n[24.191926, \"o\", \"\\b\\u001b[24mad\"]\n[24.372643, \"o\", \"d\"]\n[24.466264, \"o\", \" \"]\n[25.414876, \"o\", \"\\u001b[4ma\\u001b[24m\"]\n[25.522413, \"o\", \"\\b\\u001b[24mal\"]\n[25.687799, \"o\", \"i\"]\n[25.748435, \"o\", \"c\"]\n[25.806366, \"o\", \"e\"]\n[25.863034, \"o\", \" \"]\n[29.196254, \"o\", \"\\u001b[22D\\u001b[39mb\\u001b[39mr\\u001b[39mi\\u001b[39mg\\u001b[18C\\u001b[7mQmUzLSHCKUNDow8YS6tEZAPzKYVqoG8oYj41ioN1YgHWCb:SEfXUDA157kQoqRc6EvwRy3v3kFdTJh2HXcWso7HBGJVMi7W\\u001b[7mh\\u001b[7mCC476Gp6BHMQKtZiVuJqc4o43RZ1dz3dPeJEgLyMcRTB\\u001b[27m\\u001b[K\"]\n[30.112026, \"o\", \"\\u001b[A\\u001b[43D\\u001b[32mb\\u001b[32mr\\u001b[32mi\\u001b[32mg\\u001b[39m\\u001b[18C\\u001b[27mQ\\u001b[27mm\\u001b[27mU\\u001b[27mz\\u001b[27mL\\u001b[27mS\\u001b[27mH\\u001b[27mC\\u001b[27mK\\u001b[27mU\\u001b[27mN\\u001b[27mD\\u001b[27mo\\u001b[27mw\\u001b[27m8\\u001b[27mY\\u001b[27mS\\u001b[27m6\\u001b[27mt\\u001b[27mE\\u001b[27mZ\\u001b[27mA\\u001b[27mP\\u001b[27mz\\u001b[27mK\\u001b[27mY\\u001b[27mV\\u001b[27mq\\u001b[27mo\\u001b[27mG\\u001b[27m8\\u001b[27mo\\u001b[27mY\\u001b[27mj\\u001b[27m4\\u001b[27m1\\u001b[27mi\\u001b[27mo\\u001b[27mN\\u001b[27m1\\u001b[27mY\\u001b[27mg\\u001b[27mH\\u001b[27mW\\u001b[27mC\\u001b[27mb\\u001b[27m:\\u001b[27mS\\u001b[27mE\\u001b[27mf\\u001b[27mX\\u001b[27mU\\u001b[27mD\\u001b[27mA\\u001b[27m1\\u001b[27m5\\u001b[27m7\\u001b[27mk\\u001b[27mQ\\u001b[27mo\\u001b[27mq\\u001b[27mR\\u001b[27mc\\u001b[27m6\\u001b[27mE\\u001b[27mv\\u001b[27mw\\u001b[27mR\\u001b[27my\\u001b[27m3\\u001b[27mv\\u001b[27m3\\u001b[27mk\\u001b[27mF\\u001b[27md\\u001b[27mT\\u001b[27mJ\\u001b[27mh\\u001b[27m2\\u001b[27mH\\u001b[27mX\\u001b[27mc\\u001b[27mW\\u001b[27ms\\u001b[27mo\\u001b[27m7\\u001b[27mH\\u001b[27mB\\u001b[27mG\\u001b[27mJ\\u001b[27mV\\u001b[27mM\\u001b[27mi\\u001b[27m7\\u001b[27mWh\\u001b[27mC\\u001b[27mC\\u001b[27m4\\u001b[27m7\\u001b[27m6\\u001b[27mG\\u001b[27mp\\u001b[27m6\\u001b[27mB\\u001b[27mH\\u001b[27mM\\u001b[27mQ\\u001b[27mK\\u001b[27mt\\u001b[27mZ\\u001b[27mi\\u001b[27mV\\u001b[27mu\\u001b[27mJ\\u001b[27mq\\u001b[27mc\\u001b[27m4\\u001b[27mo\\u001b[27m4\\u001b[27m3\\u001b[27mR\\u001b[27mZ\\u001b[27m1\\u001b[27md\\u001b[27mz\\u001b[27m3\\u001b[27md\\u001b[27mP\\u001b[27me\\u001b[27mJ\\u001b[27mE\\u001b[27mg\\u001b[27mL\\u001b[27my\\u001b[27mM\\u001b[27mc\\u001b[27mR\\u001b[27mT\\u001b[27mB\"]\n[30.112723, \"o\", \"\\u001b[?1l\\u001b>\"]\n[30.121712, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[30.123729, \"o\", \"\\u001b]2;brig remote add alice \\u0007\\u001b]1;brig\\u0007\"]\n[30.245746, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[30.245919, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[30.24613, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[30.246223, \"o\", \"\\u001b[?1h\\u001b=\"]\n[30.246451, \"o\", \"\\u001b[?2004h\"]\n[31.932036, \"o\", \"\\u001b[32mbrig\\u001b[39m remote add alice QmUzLSHCKUNDow8YS6tEZAPzKYVqoG8oYj41ioN1YgHWCb:SEfXUDA157kQoqRc6EvwRy3v3kFdTJh2HXcWso7HBGJVMi7WhCC476Gp6BHMQKtZiVuJqc4o43RZ1dz3dPeJEgLyMcRTB\\u001b[K\"]\n[32.081344, \"o\", \"\\u001b[A\\u001b[31Dls\\u001b[K\\u001b[1B\\r\\u001b[K\\u001b[A\\u001b[16C\"]\n[32.635824, \"o\", \"\\u001b[?1l\\u001b>\"]\n[32.641717, \"o\", \"\\u001b[?2004l\\u001b[1B\\r\"]\n[32.643823, \"o\", \"\\u001b]2;brig remote ls\\u0007\\u001b]1;brig\\u0007\"]\n[32.753559, \"o\", \"- Name: alice\\r\\n Fingerprint: QmUzLSHCKUNDow8YS6tEZAPzKYVqoG8oYj41ioN1YgHWCb:SEfXUDA157kQoqRc6EvwRy3v3kFdTJh2HXcWso7HBGJVMi7WhCC476Gp6BHMQKtZiVuJqc4o43RZ1dz3dPeJEgLyMcRTB\\r\\n\\r\\n\"]\n[32.755086, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[32.755201, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[32.755269, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[32.755388, \"o\", \"\\u001b[?1h\\u001b=\"]\n[32.755607, \"o\", \"\\u001b[?2004h\"]\n[34.857157, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[34.964859, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[35.066833, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[35.147421, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[35.21895, \"o\", \" \"]\n[35.530163, \"o\", \"\\u001b[4mr\\u001b[24m\"]\n[35.592531, \"o\", \"\\b\\u001b[4mr\\u001b[4me\\u001b[24m\"]\n[35.64326, \"o\", \"\\b\\b\\u001b[24mr\\u001b[24mem\"]\n[35.818233, \"o\", \"o\"]\n[35.86627, \"o\", \"t\"]\n[35.909441, \"o\", \"e\"]\n[35.960071, \"o\", \" \"]\n[36.225287, \"o\", \"\\u001b[4me\\u001b[24m\"]\n[36.330022, \"o\", \"\\b\\u001b[24med\"]\n[36.445383, \"o\", \"i\"]\n[36.537795, \"o\", \"t\"]\n[36.657444, \"o\", \"\\u001b[?1l\\u001b>\"]\n[36.663347, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[36.665361, \"o\", \"\\u001b]2;brig remote edit\\u0007\\u001b]1;brig\\u0007\"]\n[36.809897, \"o\", \"\\u001b[?1000h\\u001b[?2004h\\u001b[?1049h\\u001b[22;0;0t\\u001b[?1h\\u001b=\\u001b[?2004h\"]\n[36.810124, \"o\", \"\\u001b[1;29r\\u001b[?12h\\u001b[?12l\\u001b[27m\\u001b[23m\\u001b[29m\\u001b[m\\u001b[H\\u001b[2J\\u001b[?25l\\u001b[29;1H\\\"/tmp/brig-cmd-buffer-7541559781143048777yml\\\"\"]\n[36.810195, \"o\", \" 2L, 170C\"]\n[36.811364, \"o\", \"\\u001b[2;1H▽\\u001b[6n\\u001b[2;1H \\u001b[1;1H\\u001b[>c\\u001b]10;?\\u0007\\u001b]11;?\\u0007\"]\n[36.811624, \"o\", \"\\u001b[1;1H- Name: alice\\r\\n Fingerprint: QmUzLSHCKUNDow8YS6tEZAPzKYVqoG8oYj41ioN1YgHWCb:SEfXUDA157kQoqRc6EvwRy3v3kFdTJh2HXcWso7HBGJVMi7WhCC476Gp66\\u001b[3;1HBHMQKtZiVuJqc4o43RZ1dz3dPeJEgLyMcRTB\\r\\n\\u001b[94m~ \\u001b[5;1H~ \\u001b[6;1H~ \\u001b[7;1H~ \\u001b[8;1H~ \\u001b[9;1H~ \\u001b[10;1H~ \"]\n[36.811691, \"o\", \" \\u001b[11;1H~ \\u001b[12;1H~ \\u001b[13;1H~ \\u001b[14;1H~ \\u001b[15;1H~ \\u001b[16;1H~ \\u001b[17;1H~ \\u001b[18;1H~ \"]\n[36.811752, \"o\", \" \\u001b[19;1H~ \\u001b[20;1H~ \\u001b[21;1H~ \\u001b[22;1H~ \\u001b[23;1H~ \\u001b[24;1H~ \\u001b[25;1H~ \\u001b[26;1H~ \"]\n[36.811809, \"o\", \" \\u001b[27;1H~ \\u001b[28;1H~ \\u001b[m\\u001b[29;102H1,1\\u001b[9CAlles\\u001b[1;1H\\u001b[?25h\"]\n[36.821729, \"o\", \"\\u001bP+q436f\\u001b\\\\\\u001bP+q6b75\\u001b\\\\\\u001bP+q6b64\\u001b\\\\\\u001bP+q6b72\\u001b\\\\\\u001bP+q6b6c\\u001b\\\\\\u001bP+q2332\\u001b\\\\\\u001bP+q2334\\u001b\\\\\\u001bP+q2569\\u001b\\\\\\u001bP+q2a37\\u001b\\\\\\u001bP+q6b31\\u001b\\\\\"]\n[36.824519, \"o\", \"\\u001b[?1000l\\u001b[?1006h\\u001b[?1002h\\u001b[?1006l\\u001b[?1002l\\u001b[?1006h\\u001b[?1002h\\u001b[27m\\u001b[23m\\u001b[29m\\u001b[m\\u001b[H\\u001b[2J\\u001b[?25l\\u001b[1;1H- Name: alice\\r\\n Fingerprint: QmUzLSHCKUNDow8YS6tEZAPzKYVqoG8oYj41ioN1YgHWCb:SEfXUDA157kQoqRc6EvwRy3v3kFdTJh2HXcWso7HBGJVMi7WhCC476Gp66\\u001b[3;1HBHMQKtZiVuJqc4o43RZ1dz3dPeJEgLyMcRTB\\r\\n\\u001b[94m~ \\u001b[5;1H~ \\u001b[6;1H~ \\u001b[7;1H~ \\u001b[8;1H~ \\u001b[9;1H~ \\u001b[1\"]\n[36.824903, \"o\", \"0;1H~ \\u001b[11;1H~ \\u001b[12;1H~ \\u001b[13;1H~ \\u001b[14;1H~ \\u001b[15;1H~ \\u001b[16;1H~ \\u001b[17;1H~ \\u001b[18;1H~ \"]\n[36.825159, \"o\", \" \\u001b[19;1H~ \\u001b[20;1H~ \\u001b[21;1H~ \\u001b[22;1H~ \\u001b[23;1H~ \\u001b[24;1H~ \\u001b[25;1H~ \\u001b[26;1H~ \"]\n[36.825392, \"o\", \" \\u001b[27;1H~ \\u001b[28;1H~ \\u001b[m\\u001b[29;102H1,1\\u001b[9CAlles\\r\\\"/tmp/brig-cmd-buffer-7541559781143048777yml\\\" 2L, 170C\\u001b[1;1H\\u001b[?25h\"]\n[37.72463, \"o\", \"\\u001b[?25l\\u001b[29;92HO\\u001b[1;1H\"]\n[37.724801, \"o\", \"\\u001b[29;92H \\u001b[1;1H\\u001b[29;1H\\u001b[1m-- EINFÜGEN --\\u001b[m\\u001b[29;15H\\u001b[K\\u001b[29;102H1,1\\u001b[9CAlles\"]\n[37.725003, \"o\", \"\\u001b[1;28r\\u001b[1;1H\\u001b[L\\u001b[1;29r\"]\n[37.725116, \"o\", \"\\u001b[29;102H\\u001b[K\\u001b[29;102H1,1\\u001b[9CAlles\\u001b[1;1H\\u001b[?25h\"]\n[38.227707, \"o\", \"\\u001b[?25l#\\u001b[29;104H2\\u001b[1;2H\\u001b[?25h\"]\n[38.338907, \"o\", \"\\u001b[?25l\\u001b[29;104H3\\u001b[1;3H\\u001b[?25h\"]\n[38.530171, \"o\", \"\\u001b[?25lY\\u001b[29;104H4\\u001b[1;4H\\u001b[?25h\"]\n[38.666247, \"o\", \"\\u001b[?25lo\\u001b[29;104H5\\u001b[1;5H\\u001b[?25h\"]\n[38.829278, \"o\", \"\\u001b[?25lu\\u001b[29;104H6\\u001b[1;6H\\u001b[?25h\"]\n[38.915063, \"o\", \"\\u001b[?25l\\u001b[29;104H7\\u001b[1;7H\\u001b[?25h\"]\n[39.028541, \"o\", \"\\u001b[?25lc\\u001b[29;104H8\\u001b[1;8H\\u001b[?25h\"]\n[39.105496, \"o\", \"\\u001b[?25lo\\u001b[29;104H9\\u001b[1;9H\\u001b[?25h\"]\n[39.261918, \"o\", \"\\u001b[?25lu\\u001b[29;104H10\\u001b[1;10H\\u001b[?25h\"]\n[39.427131, \"o\", \"\\u001b[?25ll\\u001b[29;105H1\\u001b[1;11H\\u001b[?25h\"]\n[39.509225, \"o\", \"\\u001b[?25ld\\u001b[29;105H2\\u001b[1;12H\\u001b[?25h\"]\n[39.598817, \"o\", \"\\u001b[?25l\\u001b[29;105H3\\u001b[1;13H\\u001b[?25h\"]\n[39.661772, \"o\", \"\\u001b[?25la\\u001b[29;105H4\\u001b[1;14H\\u001b[?25h\"]\n[40.286289, \"o\", \"\\u001b[?25ll\\u001b[29;105H5\\u001b[1;15H\\u001b[?25h\"]\n[40.445697, \"o\", \"\\u001b[?25ls\\u001b[29;105H6\\u001b[1;16H\\u001b[?25h\"]\n[40.581195, \"o\", \"\\u001b[?25lo\\u001b[29;105H7\\u001b[1;17H\\u001b[?25h\"]\n[40.693541, \"o\", \"\\u001b[?25l\\u001b[29;105H8\\u001b[1;18H\\u001b[?25h\"]\n[40.787179, \"o\", \"\\u001b[?25le\\u001b[29;105H9\\u001b[1;19H\\u001b[?25h\"]\n[40.916071, \"o\", \"\\u001b[?25ld\\u001b[29;104H20\\u001b[1;20H\\u001b[?25h\"]\n[41.055248, \"o\", \"\\u001b[?25li\\u001b[29;105H1\\u001b[1;21H\\u001b[?25h\"]\n[41.122993, \"o\", \"\\u001b[?25lt\\u001b[29;105H2\\u001b[1;22H\\u001b[?25h\"]\n[41.224271, \"o\", \"\\u001b[?25l\\u001b[29;105H3\\u001b[1;23H\\u001b[?25h\"]\n[41.297735, \"o\", \"\\u001b[?25lt\\u001b[29;105H4\\u001b[1;24H\\u001b[?25h\"]\n[41.374971, \"o\", \"\\u001b[?25lh\\u001b[29;105H5\\u001b[1;25H\\u001b[?25h\"]\n[41.499302, \"o\", \"\\u001b[?25le\\u001b[29;105H6\\u001b[1;26H\\u001b[?25h\"]\n[41.541186, \"o\", \"\\u001b[?25l\\u001b[29;105H7\\u001b[1;27H\\u001b[?25h\"]\n[41.660885, \"o\", \"\\u001b[?25ll\\u001b[29;105H8\\u001b[1;28H\\u001b[?25h\"]\n[41.801246, \"o\", \"\\u001b[?25li\\u001b[29;105H9\\u001b[1;29H\\u001b[?25h\"]\n[41.856822, \"o\", \"\\u001b[?25ls\\u001b[29;104H30\\u001b[1;30H\\u001b[?25h\"]\n[42.001292, \"o\", \"\\u001b[?25lt\\u001b[29;105H1\\u001b[1;31H\\u001b[?25h\"]\n[42.047968, \"o\", \"\\u001b[?25l\\u001b[29;105H2\\u001b[1;32H\\u001b[?25h\"]\n[42.143845, \"o\", \"\\u001b[?25li\\u001b[29;105H3\\u001b[1;33H\\u001b[?25h\"]\n[42.309877, \"o\", \"\\u001b[?25ln\\u001b[29;105H4\\u001b[1;34H\\u001b[?25h\"]\n[42.392342, \"o\", \"\\u001b[?25l\\u001b[29;105H5\\u001b[1;35H\\u001b[?25h\"]\n[42.592242, \"o\", \"\\u001b[?25lt\\u001b[29;105H6\\u001b[1;36H\\u001b[?25h\"]\n[42.668149, \"o\", \"\\u001b[?25lh\\u001b[29;105H7\\u001b[1;37H\\u001b[?25h\"]\n[42.746122, \"o\", \"\\u001b[?25le\\u001b[29;105H8\\u001b[1;38H\\u001b[?25h\"]\n[42.828254, \"o\", \"\\u001b[?25l\\u001b[29;105H9\\u001b[1;39H\\u001b[?25h\"]\n[43.003693, \"o\", \"\\u001b[?25le\\u001b[29;104H40\\u001b[1;40H\\u001b[?25h\"]\n[43.154363, \"o\", \"\\u001b[?25ld\\u001b[29;105H1\\u001b[1;41H\\u001b[?25h\"]\n[43.263772, \"o\", \"\\u001b[?25li\\u001b[29;105H2\\u001b[1;42H\\u001b[?25h\"]\n[43.367632, \"o\", \"\\u001b[?25lt\\u001b[29;105H3\\u001b[1;43H\\u001b[?25h\"]\n[43.507165, \"o\", \"\\u001b[?25lo\\u001b[29;105H4\\u001b[1;44H\\u001b[?25h\"]\n[43.57823, \"o\", \"\\u001b[?25lr\\u001b[29;105H5\\u001b[1;45H\\u001b[?25h\"]\n[43.693052, \"o\", \"\\u001b[?25l\\u001b[29;105H6\\u001b[1;46H\\u001b[?25h\"]\n[43.939149, \"o\", \"\\u001b[?25lo\\u001b[29;105H7\\u001b[1;47H\\u001b[?25h\"]\n[44.019491, \"o\", \"\\u001b[?25lf\\u001b[29;105H8\\u001b[1;48H\\u001b[?25h\"]\n[44.112539, \"o\", \"\\u001b[?25l\\u001b[29;105H9\\u001b[1;49H\\u001b[?25h\"]\n[44.20118, \"o\", \"\\u001b[?25lt\\u001b[29;104H50\\u001b[1;50H\\u001b[?25h\"]\n[44.2878, \"o\", \"\\u001b[?25lo\\u001b[29;105H1\\u001b[1;51H\\u001b[?25h\"]\n[44.42828, \"o\", \"\\u001b[?25lu\\u001b[29;105H2\\u001b[1;52H\\u001b[?25h\"]\n[44.684537, \"o\", \"\\u001b[?25l\\u001b[1;51H\\u001b[K\\u001b[29;105H1\\u001b[1;51H\\u001b[?25h\"]\n[44.830548, \"o\", \"\\u001b[?25l\\u001b[1;50H\\u001b[K\\u001b[29;105H0\\u001b[1;50H\\u001b[?25h\"]\n[44.970737, \"o\", \"\\u001b[?25l\\u001b[1;49H\\u001b[K\\u001b[29;104H49\\u001b[1;49H\\u001b[?25h\"]\n[45.046065, \"o\", \"\\u001b[?25ly\\u001b[29;104H50\\u001b[1;50H\\u001b[?25h\"]\n[45.133149, \"o\", \"\\u001b[?25lo\\u001b[29;105H1\\u001b[1;51H\\u001b[?25h\"]\n[45.287311, \"o\", \"\\u001b[?25lu\\u001b[29;105H2\\u001b[1;52H\\u001b[?25h\"]\n[45.328195, \"o\", \"\\u001b[?25lr\\u001b[29;105H3\\u001b[1;53H\\u001b[?25h\"]\n[45.402768, \"o\", \"\\u001b[?25l\\u001b[29;105H4\\u001b[1;54H\\u001b[?25h\"]\n[45.498703, \"o\", \"\\u001b[?25lc\\u001b[29;105H5\\u001b[1;55H\\u001b[?25h\"]\n[45.537462, \"o\", \"\\u001b[?25lh\\u001b[29;105H6\\u001b[1;56H\\u001b[?25h\"]\n[45.718158, \"o\", \"\\u001b[?25lo\\u001b[29;105H7\\u001b[1;57H\\u001b[?25h\"]\n[45.8428, \"o\", \"\\u001b[?25li\\u001b[29;105H8\\u001b[1;58H\\u001b[?25h\"]\n[45.905946, \"o\", \"\\u001b[?25lc\\u001b[29;105H9\\u001b[1;59H\\u001b[?25h\"]\n[45.953682, \"o\", \"\\u001b[?25le\\u001b[29;104H60\\u001b[1;60H\\u001b[?25h\"]\n[46.081636, \"o\", \"\\u001b[?25l.\\u001b[29;105H1\\u001b[1;61H\\u001b[?25h\"]\n[46.267888, \"o\", \"\\u001b[?25l.\\u001b[29;105H2\\u001b[1;62H\\u001b[?25h\"]\n[46.410391, \"o\", \"\\u001b[?25l.\\u001b[29;105H3\\u001b[1;63H\\u001b[?25h\"]\n[46.586319, \"o\", \"\\u001b[29;1H\\u001b[K\\u001b[1;62H\\u001b[?25l\\u001b[29;92H^[\\u001b[1;62H\"]\n[46.686908, \"o\", \"\\u001b[29;92H \\u001b[1;63H\"]\n[46.688225, \"o\", \"\\u001b[29;102H1,62\\u001b[8CAlles\\u001b[1;62H\\u001b[?25h\"]\n[46.886607, \"o\", \"\\u001b[?25l\\u001b[29;92H:\\u001b[1;62H\\u001b[29;92H\\u001b[K\\u001b[29;1H:\"]\n[46.886749, \"o\", \"\\u001b[?2004h\\u001b[?25h\"]\n[46.978696, \"o\", \"w\\u001b[?25l\\u001b[?25h\"]\n[47.035528, \"o\", \"q\\u001b[?25l\\u001b[?25h\"]\n[47.105184, \"o\", \"\\r\"]\n[47.105636, \"o\", \"\\u001b[?25l\\u001b[?1006l\\u001b[?1002l\\u001b[?2004l\\\"/tmp/brig-cmd-buffer-7541559781143048777yml\\\"\"]\n[47.106097, \"o\", \" 3L, 233C geschrieben\"]\n[47.108146, \"o\", \"\\r\\r\\r\\n\\u001b[?2004l\\u001b[?1l\\u001b>\\u001b[?25h\\u001b[?1049l\\u001b[23;0;0t\"]\n[47.117848, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[47.118393, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[47.118661, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[47.118889, \"o\", \"\\u001b[?1h\\u001b=\"]\n[47.119552, \"o\", \"\\u001b[?2004h\"]\n[47.797722, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[47.92976, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[48.010327, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[48.120604, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[48.19344, \"o\", \" \"]\n[48.40232, \"o\", \"n\"]\n[48.485116, \"o\", \"e\"]\n[48.582485, \"o\", \"t\"]\n[48.636681, \"o\", \" \"]\n[48.710048, \"o\", \"\\u001b[4ml\\u001b[24m\"]\n[48.858827, \"o\", \"\\b\\u001b[24mli\"]\n[48.950673, \"o\", \"s\"]\n[49.079882, \"o\", \"t\"]\n[49.184659, \"o\", \"\\u001b[?1l\\u001b>\"]\n[49.189848, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[49.192088, \"o\", \"\\u001b]2;brig net list\\u0007\\u001b]1;brig\\u0007\"]\n[49.350062, \"o\", \"NAME ADDR ROUNDTRIP LASTSEEN \\r\\nalice QmUzLSHCK 0s \\u001b[32m✔ Feb 28 16:54:59\\u001b[0m \"]\n[49.350218, \"o\", \"\\r\\n\"]\n[49.351703, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[49.351808, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\"]\n[49.351835, \"o\", \"\\u001b]1;~/dev/brig/docs\\u0007\"]\n[49.351916, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[49.352019, \"o\", \"\\u001b[?1h\\u001b=\"]\n[49.352226, \"o\", \"\\u001b[?2004h\"]\n[52.42199, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n" }, { "path": "docs/asciinema/8_sync.json", "content": "{\"version\": 2, \"width\": 119, \"height\": 29, \"timestamp\": 1519834569, \"env\": {\"SHELL\": \"/bin/zsh\", \"TERM\": \"xterm-256color\"}, \"title\": \"brig sync\"}\n[0.265949, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[0.266115, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[0.266291, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\\u001b[?1h\\u001b=\"]\n[0.266474, \"o\", \"\\u001b[?2004h\"]\n[0.900166, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[1.020164, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[1.114403, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[1.205482, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[1.285285, \"o\", \" \"]\n[1.371649, \"o\", \"n\"]\n[1.450098, \"o\", \"e\"]\n[1.551298, \"o\", \"t\"]\n[1.621169, \"o\", \" \"]\n[1.730383, \"o\", \"\\u001b[4ml\\u001b[24m\"]\n[1.866582, \"o\", \"\\b\\u001b[24mli\"]\n[1.963829, \"o\", \"s\"]\n[2.085691, \"o\", \"t\"]\n[2.256785, \"o\", \"\\u001b[?1l\\u001b>\"]\n[2.263479, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[2.266598, \"o\", \"\\u001b]2;brig net list\\u0007\"]\n[2.273975, \"o\", \"\\u001b]1;brig\\u0007\"]\n[2.428703, \"o\", \"NAME ADDR ROUNDTRIP LASTSEEN \\r\\nalice QmUzLSHCK 1ms \\u001b[32m✔ Feb 28 17:16:09\\u001b[0m \\r\\n\"]\n[2.430672, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[2.430901, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[2.431006, \"o\", \"\\u001b[?1h\\u001b=\"]\n[2.431162, \"o\", \"\\u001b[?2004h\"]\n[4.013274, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[4.158376, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[4.26778, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[4.387579, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[4.505512, \"o\", \" \"]\n[4.725358, \"o\", \"d\"]\n[4.846141, \"o\", \"i\"]\n[4.919277, \"o\", \"f\"]\n[5.060555, \"o\", \"f\"]\n[5.119819, \"o\", \" \"]\n[5.235577, \"o\", \"\\u001b[4ma\\u001b[24m\"]\n[5.362352, \"o\", \"\\b\\u001b[24mal\"]\n[5.514241, \"o\", \"i\"]\n[5.602878, \"o\", \"c\"]\n[5.668968, \"o\", \"e\"]\n[5.933904, \"o\", \"\\u001b[?1l\\u001b>\"]\n[5.938148, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[5.94028, \"o\", \"\\u001b]2;brig diff alice\\u0007\\u001b]1;brig\\u0007\"]\n[6.125933, \"o\", \"\\u001b[35m•\\u001b[0m\\r\\n├\"]\n[6.126184, \"o\", \"──\\u001b[32m + greetings_from_alice\\u001b[0m\\r\\n├──\\u001b[31m - new-file\\u001b[0m\\r\\n├──\\u001b[36m README.md ⇄ README.md\\u001b[0m\\r\\n└──\\u001b[31m - sub\\u001b[0m\\r\\n └──\\u001b[31m - music.mp3\\u001b[0m\\r\\n\\r\\n1 directory, 4 files\\r\\n\"]\n[6.127905, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[6.127995, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\"]\n[6.128022, \"o\", \"\\u001b]1;~/dev/brig/docs\\u0007\"]\n[6.128098, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[6.128194, \"o\", \"\\u001b[?1h\\u001b=\"]\n[6.128434, \"o\", \"\\u001b[?2004h\"]\n[7.746912, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[7.859958, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[7.979952, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[8.088853, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[8.174836, \"o\", \" \"]\n[8.975584, \"o\", \"s\"]\n[9.18399, \"o\", \"y\"]\n[9.343424, \"o\", \"n\"]\n[9.460907, \"o\", \"c\"]\n[9.550074, \"o\", \" \"]\n[9.684063, \"o\", \"\\u001b[4ma\\u001b[24m\"]\n[9.816329, \"o\", \"\\b\\u001b[24mal\"]\n[9.971113, \"o\", \"i\"]\n[10.076632, \"o\", \"c\"]\n[10.141374, \"o\", \"e\"]\n[10.418895, \"o\", \"\\u001b[?1l\\u001b>\"]\n[10.426116, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[10.428601, \"o\", \"\\u001b]2;brig sync alice\\u0007\\u001b]1;brig\\u0007\"]\n[10.855741, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[10.85592, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[10.856032, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[10.856137, \"o\", \"\\u001b[?1h\\u001b=\"]\n[10.856417, \"o\", \"\\u001b[?2004h\"]\n[12.084895, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[12.233591, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[12.318774, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[12.447816, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[12.534781, \"o\", \" \"]\n[13.184579, \"o\", \"\\u001b[4ml\\u001b[24m\"]\n[13.382836, \"o\", \"\\b\\u001b[4ml\\u001b[4mo\\u001b[24m\"]\n[13.521385, \"o\", \"\\b\\u001b[4mo\\u001b[4mg\\u001b[24m\"]\n[13.689946, \"o\", \"\\u001b[?1l\\u001b>\"]\n[13.691892, \"o\", \"\\u001b[?2004l\\b\\b\\b\\u001b[24ml\\u001b[24mo\\u001b[24mg\\r\\r\\n\"]\n[13.696934, \"o\", \"\\u001b]2;brig log\\u0007\"]\n[13.700079, \"o\", \"\\u001b]1;brig\\u0007\"]\n[13.88577, \"o\", \"\\u001b[32mSEfXUB9STL\\u001b[0m \\u001b[33mFeb 28 17:16:20\\u001b[0m \\u001b[31m•\\u001b[0m\\u001b[36m (curr)\\u001b[0m\\r\\n\"]\n[13.885898, \"o\", \"\\u001b[32mSEfXUEP9pV\\u001b[0m \\u001b[33mFeb 28 11:54:53\\u001b[0m Merge with alice\\u001b[36m (head)\\u001b[0m\\r\\n\\u001b[32mSEfXUBmLmQ\\u001b[0m \\u001b[33mFeb 28 10:59:36\\u001b[0m edited new-file\\u001b[36m\\u001b[0m\\r\\n\\u001b[32mSEfXUCjYxA\\u001b[0m \\u001b[33mFeb 28 10:45:21\\u001b[0m Added darth vader music\\u001b[36m\\u001b[0m\\r\\n\"]\n[13.885923, \"o\", \"\\u001b[32mSEfXUCU47p\\u001b[0m \\u001b[33mFeb 28 10:45:21\\u001b[0m Added initial README.md\\u001b[36m\\u001b[0m\\r\\n\"]\n[13.885971, \"o\", \"\\u001b[32mSEfXUCEaXL\\u001b[0m \\u001b[33mFeb 28 10:45:21\\u001b[0m initial commit\\u001b[36m (init)\\u001b[0m\\r\\n\"]\n[13.887705, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[13.887892, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[13.887972, \"o\", \"\\u001b[?1h\\u001b=\"]\n[13.888183, \"o\", \"\\u001b[?2004h\"]\n[16.401928, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[16.515067, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[16.621212, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[16.721487, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[17.42544, \"o\", \" \"]\n[17.566868, \"o\", \"\\u001b[4ml\\u001b[24m\"]\n[17.683992, \"o\", \"\\b\\u001b[24mls\"]\n[17.875134, \"o\", \"\\u001b[?1l\\u001b>\"]\n[17.879444, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[17.881636, \"o\", \"\\u001b]2;brig ls\\u0007\\u001b]1;brig\\u0007\"]\n[18.095557, \"o\", \"SIZE MODTIME PATH PIN \\r\\n458 B Feb 28 10:59:50 \\u001b[37m/README.md\\u001b[0m \"]\n[18.095745, \"o\", \" \\u001b[36m🖈\\u001b[0m \\r\\n78 B Feb 28 17:16:20 \\u001b[37m/greetings_from_alice\\u001b[0m \\u001b[36m🖈\\u001b[0m \\r\\n122 B Feb 28 11:54:47 \\u001b[37m/new-file\\u001b[0m \\u001b[36m🖈\\u001b[0m \\r\\n\\u001b[33m26 MB\\u001b[0m Feb 28 10:46:48 \\u001b[32m/sub\\u001b[0m \\u001b[36m🖈\\u001b[0m \\r\\n\"]\n[18.097525, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[18.097646, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[18.097753, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \"]\n[18.097859, \"o\", \"\\u001b[K\"]\n[18.097977, \"o\", \"\\u001b[?1h\\u001b=\"]\n[18.098284, \"o\", \"\\u001b[?2004h\"]\n[19.043151, \"o\", \"\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[19.178064, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[19.265344, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[19.37568, \"o\", \"\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[19.45086, \"o\", \" \"]\n[19.677479, \"o\", \"\\u001b[4mc\\u001b[24m\"]\n[19.758973, \"o\", \"\\b\\u001b[24mca\"]\n[19.899614, \"o\", \"t\"]\n[19.969866, \"o\", \" \"]\n[22.205985, \"o\", \"\\u001b[9D\\u001b[39mb\\u001b[39mr\\u001b[39mi\\u001b[39mg\\u001b[5C\\u001b[7m/greetings_from_alice\\u001b[27m\"]\n[23.043074, \"o\", \"\\u001b[30D\\u001b[32mb\\u001b[32mr\\u001b[32mi\\u001b[32mg\\u001b[39m\\u001b[5C\\u001b[27m/\\u001b[27mg\\u001b[27mr\\u001b[27me\\u001b[27me\\u001b[27mt\\u001b[27mi\\u001b[27mn\\u001b[27mg\\u001b[27ms\\u001b[27m_\\u001b[27mf\\u001b[27mr\\u001b[27mo\\u001b[27mm\\u001b[27m_\\u001b[27ma\\u001b[27ml\\u001b[27mi\\u001b[27mc\\u001b[27me\"]\n[23.043568, \"o\", \"\\u001b[?1l\\u001b>\"]\n[23.050362, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[23.052589, \"o\", \"\\u001b]2;brig cat /greetings_from_alice\\u0007\\u001b]1;brig\\u0007\"]\n[23.223224, \"o\", \"This files comes from alice! Hi there!\\r\\n\"]\n[23.224979, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[23.225084, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig/docs\\u0007\\u001b]1;~/dev/brig/docs\\u0007\"]\n[23.225227, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[23.225277, \"o\", \"\\u001b[?1h\\u001b=\"]\n[23.225477, \"o\", \"\\u001b[?2004h\"]\n[25.956984, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n" }, { "path": "docs/asciinema/9_pin.json", "content": "{\"version\": 2, \"width\": 211, \"height\": 54, \"timestamp\": 1523915438, \"env\": {\"SHELL\": \"/bin/zsh\", \"TERM\": \"xterm-256color\"}, \"title\": \"brig pin\"}\n[0.30112, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[0.301615, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig\\u0007\\u001b]1;~/dev/brig\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[0.3017, \"o\", \"\\u001b[?1h\\u001b=\"]\n[0.301905, \"o\", \"\\u001b[?2004h\"]\n[0.838565, \"o\", \"\\u001b[4mb\\u001b[24m\"]\n[0.951131, \"o\", \"\\b\\u001b[4mb\\u001b[4mr\\u001b[24m\"]\n[1.039016, \"o\", \"\\b\\b\\u001b[4mb\\u001b[4mr\\u001b[4mi\\u001b[24m\"]\n[1.132231, \"o\", \"\\b\\b\\b\\u001b[24m\\u001b[32mb\\u001b[24m\\u001b[32mr\\u001b[24m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[1.210328, \"o\", \" \"]\n[1.313066, \"o\", \"\\u001b[4ml\\u001b[24m\"]\n[1.400234, \"o\", \"\\b\\u001b[24mls\"]\n[1.71358, \"o\", \"\\u001b[?1l\\u001b>\"]\n[1.718911, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[1.72261, \"o\", \"\\u001b]2;brig ls\\u0007\\u001b]1;brig\\u0007\"]\n[1.946186, \"o\", \"SIZE MODTIME USER PATH PIN \\r\\n649 B Apr 16 19:04:01 \\u001b[32malice\\u001b[0m \"]\n[1.946235, \"o\", \"\\u001b[37m/README.md\\u001b[0m \\u001b[36m🖈\\u001b[0m \\r\\n649 B Apr 16 23:04:58 \\u001b[32mbob\\u001b[0m \\u001b[37m/README.md.conflict.0\\u001b[0m \\r\\n16 B Apr 16 23:04:58 \\u001b[32mbob\\u001b[0m \\u001b[37m/greetings_from_bob\\u001b[0m \\u001b[36m🖈\\u001b[0m \\r\\n0 B Apr 16 19:06:32 \\u001b[32malice\\u001b[0m \\u001b[37m/new-file\\u001b[0m \\u001b[36m🖈\\u001b[0m \\r\\n\"]\n[1.946464, \"o\", \"0 B Apr 16 19:06:51 \\u001b[32malice\\u001b[0m \\u001b[32m/sub\\u001b[0m \\r\\n\"]\n[1.947875, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[1.947981, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig\\u0007\"]\n[1.948001, \"o\", \"\\u001b]1;~/dev/brig\\u0007\"]\n[1.948121, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[1.948181, \"o\", \"\\u001b[?1h\\u001b=\"]\n[1.948385, \"o\", \"\\u001b[?2004h\"]\n[3.11197, \"o\", \"\\u001b[4mb\\u001b[24m\"]\n[3.241792, \"o\", \"\\b\\u001b[4mb\\u001b[4mr\\u001b[24m\"]\n[3.342311, \"o\", \"\\b\\b\\u001b[4mb\\u001b[4mr\\u001b[4mi\\u001b[24m\"]\n[3.437441, \"o\", \"\\b\\b\\b\\u001b[24m\\u001b[32mb\\u001b[24m\\u001b[32mr\\u001b[24m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[3.506383, \"o\", \" \"]\n[3.695927, \"o\", \"p\"]\n[3.862273, \"o\", \"i\"]\n[4.035358, \"o\", \"n\"]\n[4.113841, \"o\", \" \"]\n[4.243842, \"o\", \"\\u001b[4mn\\u001b[24m\"]\n[4.292819, \"o\", \"\\b\\u001b[4mn\\u001b[4me\\u001b[24m\"]\n[4.46478, \"o\", \"\\b\\b\\u001b[24mn\\u001b[24mew\"]\n[4.563586, \"o\", \"-\"]\n[4.66959, \"o\", \"f\"]\n[4.756648, \"o\", \"i\"]\n[4.912451, \"o\", \"l\"]\n[4.997947, \"o\", \"e\"]\n[5.161257, \"o\", \"\\u001b[?1l\\u001b>\"]\n[5.176626, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[5.17932, \"o\", \"\\u001b]2;brig pin new-file\\u0007\\u001b]1;brig\\u0007\"]\n[5.353915, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[5.354034, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig\\u0007\\u001b]1;~/dev/brig\\u0007\"]\n[5.354255, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\\u001b[?1h\\u001b=\"]\n[5.354405, \"o\", \"\\u001b[?2004h\"]\n[5.908839, \"o\", \"\\u001b[4mb\\u001b[24m\"]\n[6.051314, \"o\", \"\\b\\u001b[4mb\\u001b[4mr\\u001b[24m\"]\n[6.104361, \"o\", \"\\b\\b\\u001b[4mb\\u001b[4mr\\u001b[4mi\\u001b[24m\"]\n[6.224113, \"o\", \"\\b\\b\\b\\u001b[24m\\u001b[32mb\\u001b[24m\\u001b[32mr\\u001b[24m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[6.25579, \"o\", \" \"]\n[6.353504, \"o\", \"\\u001b[4ml\\u001b[24m\"]\n[6.443741, \"o\", \"\\b\\u001b[24mls\"]\n[6.647207, \"o\", \"\\u001b[?1l\\u001b>\"]\n[6.653213, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[6.65678, \"o\", \"\\u001b]2;brig ls\\u0007\\u001b]1;brig\\u0007\"]\n[6.841293, \"o\", \"SIZE MODTIME USER PATH PIN \\r\\n649 B Apr 16 19:04:01 \\u001b[32malice\\u001b[0m \\u001b[37m/README.md\\u001b[0m \"]\n[6.841343, \"o\", \" \\u001b[36m🖈\\u001b[0m \\r\\n649 B Apr 16 23:04:58 \\u001b[32mbob\\u001b[0m \\u001b[37m/README.md.conflict.0\\u001b[0m \\r\\n16 B Apr 16 23:04:58 \\u001b[32mbob\\u001b[0m \\u001b[37m/greetings_from_bob\\u001b[0m \\u001b[36m🖈\\u001b[0m \\r\\n0 B Apr 16 19:06:32 \\u001b[32malice\\u001b[0m \\u001b[37m/new-file\\u001b[0m \\u001b[35m🖈\\u001b[0m \\r\\n\"]\n[6.841357, \"o\", \"0 B Apr 16 19:06:51 \\u001b[32malice\\u001b[0m \\u001b[32m/sub\\u001b[0m \"]\n[6.842937, \"o\", \" \\r\\n\"]\n[6.843057, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[6.843133, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig\\u0007\"]\n[6.843185, \"o\", \"\\u001b]1;~/dev/brig\\u0007\"]\n[6.843235, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[6.843335, \"o\", \"\\u001b[?1h\\u001b=\"]\n[6.843541, \"o\", \"\\u001b[?2004h\"]\n[8.067965, \"o\", \"\\u001b[4mb\\u001b[24m\"]\n[8.257929, \"o\", \"\\b\\u001b[4mb\\u001b[4mr\\u001b[24m\"]\n[8.386808, \"o\", \"\\b\\b\\u001b[4mb\\u001b[4mr\\u001b[4mi\\u001b[24m\"]\n[8.490726, \"o\", \"\\b\\b\\b\\u001b[24m\\u001b[32mb\\u001b[24m\\u001b[32mr\\u001b[24m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[8.574755, \"o\", \" \"]\n[9.357392, \"o\", \"p\"]\n[9.579199, \"o\", \"i\"]\n[9.77668, \"o\", \"n\"]\n[10.012016, \"o\", \" \"]\n[10.622499, \"o\", \"\\u001b[4mr\\u001b[24m\"]\n[10.728788, \"o\", \"\\b\\u001b[24mrm\"]\n[10.801391, \"o\", \" \"]\n[11.527164, \"o\", \"g\"]\n[11.750003, \"o\", \"r\"]\n[11.845988, \"o\", \"e\"]\n[12.906214, \"o\", \"e\"]\n[13.082861, \"o\", \"t\"]\n[13.201274, \"o\", \"i\"]\n[13.364602, \"o\", \"n\"]\n[13.417331, \"o\", \"g\"]\n[13.61018, \"o\", \"s\"]\n[13.814561, \"o\", \"_\"]\n[13.915141, \"o\", \"f\"]\n[14.06849, \"o\", \"r\"]\n[14.138404, \"o\", \"o\"]\n[14.289591, \"o\", \"m\"]\n[14.508238, \"o\", \"_\"]\n[14.687799, \"o\", \"b\"]\n[14.758077, \"o\", \"o\"]\n[14.86678, \"o\", \"b\"]\n[15.159533, \"o\", \"\\u001b[?1l\\u001b>\"]\n[15.195429, \"o\", \"\\u001b[?2004l\"]\n[15.196049, \"o\", \"\\r\\r\\n\"]\n[15.200924, \"o\", \"\\u001b]2;brig pin rm greetings_from_bob\\u0007\"]\n[15.202633, \"o\", \"\\u001b]1;brig\\u0007\"]\n[15.417428, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[15.417661, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig\\u0007\\u001b]1;~/dev/brig\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[15.417747, \"o\", \"\\u001b[?1h\\u001b=\"]\n[15.417951, \"o\", \"\\u001b[?2004h\"]\n[15.962116, \"o\", \"\\u001b[4mb\\u001b[24m\"]\n[16.083125, \"o\", \"\\b\\u001b[4mb\\u001b[4mr\\u001b[24m\"]\n[16.170519, \"o\", \"\\b\\b\\u001b[4mb\\u001b[4mr\\u001b[4mi\\u001b[24m\"]\n[16.260992, \"o\", \"\\b\\b\\b\\u001b[24m\\u001b[32mb\\u001b[24m\\u001b[32mr\\u001b[24m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[16.298779, \"o\", \" \"]\n[16.420998, \"o\", \"\\u001b[4ml\\u001b[24m\"]\n[16.508748, \"o\", \"\\b\\u001b[24mls\"]\n[16.636075, \"o\", \"\\u001b[?1l\\u001b>\"]\n[16.64118, \"o\", \"\\u001b[?2004l\"]\n[16.641957, \"o\", \"\\r\\r\\n\"]\n[16.646004, \"o\", \"\\u001b]2;brig ls\\u0007\\u001b]1;brig\\u0007\"]\n[16.831002, \"o\", \"SIZE MODTIME USER PATH PIN \\r\\n649 B Apr 16 19:04:01 \\u001b[32malice\\u001b[0m\"]\n[16.831261, \"o\", \" \\u001b[37m/README.md\\u001b[0m \\u001b[36m🖈\\u001b[0m \\r\\n649 B Apr 16 23:04:58 \\u001b[32mbob\\u001b[0m \\u001b[37m/README.md.conflict.0\\u001b[0m \\r\\n16 B Apr 16 23:04:58 \\u001b[32mbob\\u001b[0m \\u001b[37m/greetings_from_bob\\u001b[0m \\r\\n0 B Apr 16 19:06:32 \\u001b[32malice\\u001b[0m \\u001b[37m/new-file\\u001b[0m \\u001b[35m🖈\\u001b[0m \\r\\n0 B Apr 16 19:06:51 \\u001b[32malice\\u001b[0m \\u001b[32m/sub\\u001b[0m \\r\\n\"]\n[16.832584, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[16.832682, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig\\u0007\"]\n[16.832708, \"o\", \"\\u001b]1;~/dev/brig\\u0007\"]\n[16.832772, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[16.832864, \"o\", \"\\u001b[?1h\\u001b=\"]\n[16.833068, \"o\", \"\\u001b[?2004h\"]\n[18.59478, \"o\", \"\\u001b[4mb\\u001b[24m\"]\n[18.715649, \"o\", \"\\b\\u001b[4mb\\u001b[4mr\\u001b[24m\"]\n[18.852998, \"o\", \"\\b\\b\\u001b[4mb\\u001b[4mr\\u001b[4mi\\u001b[24m\"]\n[18.967583, \"o\", \"\\b\\b\\b\\u001b[24m\\u001b[32mb\\u001b[24m\\u001b[32mr\\u001b[24m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[19.0639, \"o\", \" \"]\n[20.2069, \"o\", \"g\"]\n[20.395469, \"o\", \"c\"]\n[20.53824, \"o\", \" \"]\n[21.301441, \"o\", \"\\u001b[?1l\\u001b>\"]\n[21.307104, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[21.311895, \"o\", \"\\u001b]2;brig gc\\u0007\\u001b]1;brig\\u0007\"]\n[21.502859, \"o\", \"CONTENT HASH OWNER \\r\\n\\u001b[37mQmUVb1XejZP6ENNzLgSmtobfMMALXc2GNGDTq3Q2ZXKyHN\\u001b[0m \\u001b[31mQmUVb1XejZ\\u001b[0m \\u001b[36malice\\u001b[0m \\r\\n\\u001b[37mQmUVb1XejZP6ENNzLgSmtobfMMALXc2GNGDTq3Q2ZXKyHN\\u001b[0m \"]\n[21.502908, \"o\", \"\\u001b[31mQmUVb1XejZ\\u001b[0m \\u001b[36mbob\\u001b[0m \\r\\n\"]\n[21.504292, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[21.504399, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig\\u0007\\u001b]1;~/dev/brig\\u0007\"]\n[21.504475, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[21.504573, \"o\", \"\\u001b[?1h\\u001b=\"]\n[21.50493, \"o\", \"\\u001b[?2004h\"]\n[23.327619, \"o\", \"\\u001b[4mb\\u001b[24m\"]\n[23.472565, \"o\", \"\\b\\u001b[4mb\\u001b[4mr\\u001b[24m\"]\n[23.606627, \"o\", \"\\b\\b\\u001b[4mb\\u001b[4mr\\u001b[4mi\\u001b[24m\"]\n[23.688013, \"o\", \"\\b\\b\\b\\u001b[24m\\u001b[32mb\\u001b[24m\\u001b[32mr\\u001b[24m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[23.772934, \"o\", \" \"]\n[23.901943, \"o\", \"\\u001b[4mc\\u001b[24m\"]\n[23.967451, \"o\", \"\\b\\u001b[4mc\\u001b[4ma\\u001b[24m\"]\n[24.107524, \"o\", \"\\b\\u001b[4ma\\u001b[4mt\\u001b[24m\"]\n[24.209351, \"o\", \"\\b\\b\\b\\u001b[24mc\\u001b[24ma\\u001b[24mt \"]\n[24.355964, \"o\", \"g\"]\n[24.518603, \"o\", \"r\"]\n[24.58132, \"o\", \"e\"]\n[24.758268, \"o\", \"e\"]\n[24.962315, \"o\", \"t\"]\n[25.059056, \"o\", \"i\"]\n[25.211339, \"o\", \"n\"]\n[25.25484, \"o\", \"g\"]\n[25.426998, \"o\", \"s\"]\n[25.750634, \"o\", \"_\"]\n[25.874119, \"o\", \"f\"]\n[26.013535, \"o\", \"r\"]\n[26.089798, \"o\", \"o\"]\n[26.251522, \"o\", \"m\"]\n[26.465614, \"o\", \"_\"]\n[26.608815, \"o\", \"b\"]\n[26.682756, \"o\", \"o\"]\n[26.768963, \"o\", \"b\"]\n[26.902155, \"o\", \"\\u001b[?1l\\u001b>\"]\n[26.908719, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[26.911227, \"o\", \"\\u001b]2;brig cat greetings_from_bob\\u0007\\u001b]1;brig\\u0007\"]\n[27.10136, \"o\", \"Hello from Bob!\"]\n[27.101676, \"o\", \"\\r\\n\"]\n[27.106601, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[27.106983, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig\\u0007\\u001b]1;~/dev/brig\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[27.107104, \"o\", \"\\u001b[?1h\\u001b=\"]\n[27.107364, \"o\", \"\\u001b[?2004h\"]\n[28.818037, \"o\", \"\\u001b[4mb\\u001b[24m\"]\n[29.724845, \"o\", \"\\b\\u001b[24m\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mo\\u001b[0m\\u001b[39m\"]\n[29.89817, \"o\", \"\\b\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mo\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[30.07675, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31m-\\u001b[0m\\u001b[39m\"]\n[30.309077, \"o\", \"\\b\\u001b[1m\\u001b[31m-\\u001b[1m\\u001b[31mb\\u001b[0m\\u001b[39m\"]\n[30.402236, \"o\", \"\\b\\u001b[1m\\u001b[31mb\\u001b[1m\\u001b[31mr\\u001b[0m\\u001b[39m\"]\n[30.492736, \"o\", \"\\b\\u001b[1m\\u001b[31mr\\u001b[1m\\u001b[31mi\\u001b[0m\\u001b[39m\"]\n[30.582505, \"o\", \"\\b\\b\\b\\b\\b\\b\\b\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mo\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32m-\\u001b[0m\\u001b[32mb\\u001b[0m\\u001b[32mr\\u001b[0m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[30.652536, \"o\", \" \"]\n[31.687412, \"o\", \"\\u001b[4md\\u001b[24m\"]\n[31.740827, \"o\", \"\\b\\u001b[24mda\"]\n[31.855845, \"o\", \"e\"]\n[31.950756, \"o\", \"m\"]\n[32.118484, \"o\", \"o\"]\n[32.282845, \"o\", \"n\"]\n[32.383776, \"o\", \" \"]\n[32.448042, \"o\", \"q\"]\n[32.519802, \"o\", \"u\"]\n[32.703519, \"o\", \"i\"]\n[32.71348, \"o\", \"o\"]\n[32.794122, \"o\", \"t\"]\n[33.108008, \"o\", \"\\b \\b\"]\n[33.244054, \"o\", \"\\b \\b\"]\n[33.439531, \"o\", \"t\"]\n[33.591015, \"o\", \"\\u001b[?1l\\u001b>\"]\n[33.598721, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[33.601482, \"o\", \"\\u001b]2;brig -p 6667 daemon quit\\u0007\\u001b]1;bob-brig\\u0007\"]\n[33.791989, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[33.792127, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig\\u0007\\u001b]1;~/dev/brig\\u0007\"]\n[33.792189, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[33.79226, \"o\", \"\\u001b[?1h\\u001b=\"]\n[33.792466, \"o\", \"\\u001b[?2004h\"]\n[35.253852, \"o\", \"\\u001b[32mbob-brig\\u001b[39m daemon quit\"]\n[35.462572, \"o\", \"\\u001b[20D\\u001b[32mb\\u001b[32mr\\u001b[32mi\\u001b[32mg\\u001b[39m\\u001b[39m \\u001b[39mc\\u001b[39ma\\u001b[39mt greetings_from_bob\"]\n[35.946018, \"o\", \"\\u001b[22Dgc \\u001b[19D\"]\n[36.768454, \"o\", \"\\u001b[?1l\\u001b>\"]\n[36.776677, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[36.778738, \"o\", \"\\u001b]2;brig gc\\u0007\\u001b]1;brig\\u0007\"]\n[36.95629, \"o\", \"CONTENT HASH OWNER \\r\\n\\u001b[37mQmUVb1XejZP6ENNzLgSmtobfMMALXc2GNGDTq3Q2ZXKyHN\\u001b[0m \\u001b[31mQmUVb1XejZ\\u001b[0m \\u001b[36malice\\u001b[0m \\r\\n\\u001b[37mQmUVb1XejZP6ENNzLgSmtobfMMALXc2GNGDTq3Q2ZXKyHN\\u001b[0m\"]\n[36.956342, \"o\", \" \\u001b[31mQmUVb1XejZ\\u001b[0m \\u001b[36mbob\\u001b[0m \\r\\n\"]\n[36.957825, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[36.957921, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig\\u0007\"]\n[36.957939, \"o\", \"\\u001b]1;~/dev/brig\\u0007\"]\n[36.95801, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[36.958121, \"o\", \"\\u001b[?1h\\u001b=\"]\n[36.958343, \"o\", \"\\u001b[?2004h\"]\n[37.86989, \"o\", \"\\u001b[4mb\\u001b[24m\"]\n[37.975293, \"o\", \"\\b\\u001b[4mb\\u001b[4mr\\u001b[24m\"]\n[38.06847, \"o\", \"\\b\\b\\u001b[4mb\\u001b[4mr\\u001b[4mi\\u001b[24m\"]\n[38.16544, \"o\", \"\\b\\b\\b\\u001b[24m\\u001b[32mb\\u001b[24m\\u001b[32mr\\u001b[24m\\u001b[32mi\\u001b[32mg\\u001b[39m\"]\n[38.581496, \"o\", \" \"]\n[38.986283, \"o\", \"gc \"]\n[39.402656, \"o\", \"\\b\\b\\b \\b\\b\"]\n[39.492823, \"o\", \"\\u001b[4mc\\u001b[24m\"]\n[39.56276, \"o\", \"\\b\\u001b[4mc\\u001b[4ma\\u001b[24m\"]\n[39.735627, \"o\", \"\\b\\u001b[4ma\\u001b[4mt\\u001b[24m\"]\n[39.823745, \"o\", \"\\b\\b\\b\\u001b[24mc\\u001b[24ma\\u001b[24mt \"]\n[39.936504, \"o\", \"greetings_from_bob\"]\n[40.746588, \"o\", \"\\u001b[?1l\\u001b>\"]\n[40.755815, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[40.759163, \"o\", \"\\u001b]2;brig cat greetings_from_bob\\u0007\\u001b]1;brig\\u0007\"]\n[42.622192, \"o\", \"\\r\\n\"]\n[42.782638, \"o\", \"\\r\\n\"]\n[42.952416, \"o\", \"\\r\\n\"]\n[43.167917, \"o\", \"\\r\\n\"]\n[43.649776, \"o\", \"^C\"]\n[43.659072, \"o\", \"\\r\\n\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\\u001b]2;sahib@werkbank: ~/dev/brig\\u0007\\u001b]1;~/dev/brig\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \"]\n[43.660243, \"o\", \"\\u001b[K\"]\n[43.661344, \"o\", \"\\u001b[?1h\\u001b=\"]\n[43.668262, \"o\", \"\\u001b[?2004h\"]\n[44.189165, \"o\", \"\\u001b[1m\\u001b[30m#\\u001b[0m\\u001b[39m\"]\n[44.301126, \"o\", \"\\b\\u001b[1m\\u001b[30m#\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[44.585404, \"o\", \"\\b\\b\\u001b[1m\\u001b[30m#\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mN\\u001b[0m\\u001b[39m\"]\n[45.196173, \"o\", \"\\b\\u001b[1m\\u001b[30mN\\u001b[1m\\u001b[30mo\\u001b[0m\\u001b[39m\"]\n[46.24694, \"o\", \"\\b\\u001b[1m\\u001b[30mo\\u001b[1m\\u001b[30mt\\u001b[0m\\u001b[39m\"]\n[46.33817, \"o\", \"\\b\\u001b[1m\\u001b[30mt\\u001b[1m\\u001b[30mh\\u001b[0m\\u001b[39m\"]\n[46.434192, \"o\", \"\\b\\u001b[1m\\u001b[30mh\\u001b[1m\\u001b[30mi\\u001b[0m\\u001b[39m\"]\n[46.447753, \"o\", \"\\b\\u001b[1m\\u001b[30mi\\u001b[1m\\u001b[30mo\\u001b[0m\\u001b[39m\"]\n[46.594595, \"o\", \"\\b\\u001b[1m\\u001b[30mo\\u001b[1m\\u001b[30mn\\u001b[0m\\u001b[39m\"]\n[46.707113, \"o\", \"\\b\\u001b[1m\\u001b[30mn\\u001b[1m\\u001b[30mg\\u001b[0m\\u001b[39m\"]\n[46.809953, \"o\", \"\\b\\u001b[1m\\u001b[30mg\\u001b[1m\\u001b[30m.\\u001b[0m\\u001b[39m\"]\n[46.927586, \"o\", \"\\b\\u001b[1m\\u001b[30m.\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[47.11099, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mT\\u001b[0m\\u001b[39m\"]\n[47.211523, \"o\", \"\\b\\u001b[1m\\u001b[30mT\\u001b[1m\\u001b[30mh\\u001b[0m\\u001b[39m\"]\n[47.279516, \"o\", \"\\b\\u001b[1m\\u001b[30mh\\u001b[1m\\u001b[30me\\u001b[0m\\u001b[39m\"]\n[47.357467, \"o\", \"\\b\\u001b[1m\\u001b[30me\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[47.428878, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mf\\u001b[0m\\u001b[39m\"]\n[47.533471, \"o\", \"\\b\\u001b[1m\\u001b[30mf\\u001b[1m\\u001b[30mi\\u001b[0m\\u001b[39m\"]\n[47.680787, \"o\", \"\\b\\u001b[1m\\u001b[30mi\\u001b[1m\\u001b[30ml\\u001b[0m\\u001b[39m\"]\n[47.727994, \"o\", \"\\b\\u001b[1m\\u001b[30ml\\u001b[1m\\u001b[30me\\u001b[0m\\u001b[39m\"]\n[47.825724, \"o\", \"\\b\\u001b[1m\\u001b[30me\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[47.910826, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mw\\u001b[0m\\u001b[39m\"]\n[47.969872, \"o\", \"\\b\\u001b[1m\\u001b[30mw\\u001b[1m\\u001b[30ma\\u001b[0m\\u001b[39m\"]\n[48.07492, \"o\", \"\\b\\u001b[1m\\u001b[30ma\\u001b[1m\\u001b[30ms\\u001b[0m\\u001b[39m\"]\n[48.119166, \"o\", \"\\b\\u001b[1m\\u001b[30ms\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[48.245168, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mc\\u001b[0m\\u001b[39m\"]\n[48.345909, \"o\", \"\\b\\u001b[1m\\u001b[30mc\\u001b[1m\\u001b[30ml\\u001b[0m\\u001b[39m\"]\n[48.45448, \"o\", \"\\b\\u001b[1m\\u001b[30ml\\u001b[1m\\u001b[30me\\u001b[0m\\u001b[39m\"]\n[48.525043, \"o\", \"\\b\\u001b[1m\\u001b[30me\\u001b[1m\\u001b[30ma\\u001b[0m\\u001b[39m\"]\n[48.598672, \"o\", \"\\b\\u001b[1m\\u001b[30ma\\u001b[1m\\u001b[30mn\\u001b[0m\\u001b[39m\"]\n[48.773518, \"o\", \"\\b\\u001b[1m\\u001b[30mn\\u001b[1m\\u001b[30me\\u001b[0m\\u001b[39m\"]\n[48.854201, \"o\", \"\\b\\u001b[1m\\u001b[30me\\u001b[1m\\u001b[30md\\u001b[0m\\u001b[39m\"]\n[48.916214, \"o\", \"\\b\\u001b[1m\\u001b[30md\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[49.069317, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mu\\u001b[0m\\u001b[39m\"]\n[49.479573, \"o\", \"\\b\\u001b[1m\\u001b[30mu\\u001b[1m\\u001b[30mp\\u001b[0m\\u001b[39m\"]\n[49.608735, \"o\", \"\\b\\u001b[1m\\u001b[30mp\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[49.895406, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mt\\u001b[0m\\u001b[39m\"]\n[49.996553, \"o\", \"\\b\\u001b[1m\\u001b[30mt\\u001b[1m\\u001b[30mh\\u001b[0m\\u001b[39m\"]\n[50.392309, \"o\", \"\\b\\b\\u001b[1m\\u001b[30mt\\u001b[0m\\u001b[39m\\u001b[0m\\u001b[39m \\b\"]\n[50.526275, \"o\", \"\\b\\b\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\\u001b[0m\\u001b[39m \\b\"]\n[50.712234, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mb\\u001b[0m\\u001b[39m\"]\n[50.808566, \"o\", \"\\b\\u001b[1m\\u001b[30mb\\u001b[1m\\u001b[30my\\u001b[0m\\u001b[39m\"]\n[50.884775, \"o\", \"\\b\\u001b[1m\\u001b[30my\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[51.053816, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mg\\u001b[0m\\u001b[39m\"]\n[51.263847, \"o\", \"\\b\\u001b[1m\\u001b[30mg\\u001b[1m\\u001b[30mc\\u001b[0m\\u001b[39m\"]\n[51.453123, \"o\", \"\\b\\u001b[1m\\u001b[30mc\\u001b[1m\\u001b[30m.\\u001b[0m\\u001b[39m\"]\n[51.773268, \"o\", \"\\u001b[?1l\\u001b>\"]\n[51.776168, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[51.779073, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[51.780013, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig\\u0007\\u001b]1;~/dev/brig\\u0007\"]\n[51.780404, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[51.780798, \"o\", \"\\u001b[?1h\\u001b=\"]\n[51.782864, \"o\", \"\\u001b[?2004h\"]\n[52.500575, \"o\", \"\\u001b[1m\\u001b[30m#\\u001b[0m\\u001b[39m\"]\n[52.615768, \"o\", \"\\b\\u001b[1m\\u001b[30m#\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[54.416078, \"o\", \"\\b\\b\\u001b[1m\\u001b[30m#\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mA\\u001b[0m\\u001b[39m\"]\n[54.61427, \"o\", \"\\b\\u001b[1m\\u001b[30mA\\u001b[1m\\u001b[30mn\\u001b[0m\\u001b[39m\"]\n[54.697741, \"o\", \"\\b\\u001b[1m\\u001b[30mn\\u001b[1m\\u001b[30md\\u001b[0m\\u001b[39m\"]\n[54.773046, \"o\", \"\\b\\u001b[1m\\u001b[30md\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[54.930814, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30ms\\u001b[0m\\u001b[39m\"]\n[55.041847, \"o\", \"\\b\\u001b[1m\\u001b[30ms\\u001b[1m\\u001b[30mi\\u001b[0m\\u001b[39m\"]\n[55.208351, \"o\", \"\\b\\u001b[1m\\u001b[30mi\\u001b[1m\\u001b[30mn\\u001b[0m\\u001b[39m\"]\n[55.244913, \"o\", \"\\b\\u001b[1m\\u001b[30mn\\u001b[1m\\u001b[30mc\\u001b[0m\\u001b[39m\"]\n[55.306743, \"o\", \"\\b\\u001b[1m\\u001b[30mc\\u001b[1m\\u001b[30me\\u001b[0m\\u001b[39m\"]\n[55.347446, \"o\", \"\\b\\u001b[1m\\u001b[30me\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[55.496123, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mw\\u001b[0m\\u001b[39m\"]\n[55.647841, \"o\", \"\\b\\u001b[1m\\u001b[30mw\\u001b[1m\\u001b[30me\\u001b[0m\\u001b[39m\"]\n[55.718209, \"o\", \"\\b\\u001b[1m\\u001b[30me\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[55.863195, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mq\\u001b[0m\\u001b[39m\"]\n[55.95291, \"o\", \"\\b\\u001b[1m\\u001b[30mq\\u001b[1m\\u001b[30mu\\u001b[0m\\u001b[39m\"]\n[56.111735, \"o\", \"\\b\\u001b[1m\\u001b[30mu\\u001b[1m\\u001b[30mi\\u001b[0m\\u001b[39m\"]\n[56.195887, \"o\", \"\\b\\u001b[1m\\u001b[30mi\\u001b[1m\\u001b[30mt\\u001b[0m\\u001b[39m\"]\n[56.265323, \"o\", \"\\b\\u001b[1m\\u001b[30mt\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[56.448391, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mt\\u001b[0m\\u001b[39m\"]\n[56.56379, \"o\", \"\\b\\u001b[1m\\u001b[30mt\\u001b[1m\\u001b[30mh\\u001b[0m\\u001b[39m\"]\n[56.612261, \"o\", \"\\b\\u001b[1m\\u001b[30mh\\u001b[1m\\u001b[30me\\u001b[0m\\u001b[39m\"]\n[56.699109, \"o\", \"\\b\\u001b[1m\\u001b[30me\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[57.362277, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mi\\u001b[0m\\u001b[39m\"]\n[57.522546, \"o\", \"\\b\\u001b[1m\\u001b[30mi\\u001b[1m\\u001b[30mn\\u001b[0m\\u001b[39m\"]\n[57.587003, \"o\", \"\\b\\u001b[1m\\u001b[30mn\\u001b[1m\\u001b[30ms\\u001b[0m\\u001b[39m\"]\n[57.715618, \"o\", \"\\b\\u001b[1m\\u001b[30ms\\u001b[1m\\u001b[30mt\\u001b[0m\\u001b[39m\"]\n[57.83796, \"o\", \"\\b\\u001b[1m\\u001b[30mt\\u001b[1m\\u001b[30ma\\u001b[0m\\u001b[39m\"]\n[57.915041, \"o\", \"\\b\\u001b[1m\\u001b[30ma\\u001b[1m\\u001b[30mn\\u001b[0m\\u001b[39m\"]\n[57.98336, \"o\", \"\\b\\u001b[1m\\u001b[30mn\\u001b[1m\\u001b[30mc\\u001b[0m\\u001b[39m\"]\n[58.052785, \"o\", \"\\b\\u001b[1m\\u001b[30mc\\u001b[1m\\u001b[30me\\u001b[0m\\u001b[39m\"]\n[58.129937, \"o\", \"\\b\\u001b[1m\\u001b[30me\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[58.237867, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mo\\u001b[0m\\u001b[39m\"]\n[58.367743, \"o\", \"\\b\\u001b[1m\\u001b[30mo\\u001b[1m\\u001b[30mf\\u001b[0m\\u001b[39m\"]\n[58.45754, \"o\", \"\\b\\u001b[1m\\u001b[30mf\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[58.589279, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mb\\u001b[0m\\u001b[39m\"]\n[58.673972, \"o\", \"\\b\\u001b[1m\\u001b[30mb\\u001b[1m\\u001b[30mo\\u001b[0m\\u001b[39m\"]\n[58.769003, \"o\", \"\\b\\u001b[1m\\u001b[30mo\\u001b[1m\\u001b[30mb\\u001b[0m\\u001b[39m\"]\n[58.898781, \"o\", \"\\b\\u001b[1m\\u001b[30mb\\u001b[1m\\u001b[30m,\\u001b[0m\\u001b[39m\"]\n[58.994803, \"o\", \"\\b\\u001b[1m\\u001b[30m,\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[59.157135, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mw\\u001b[0m\\u001b[39m\"]\n[59.332243, \"o\", \"\\b\\u001b[1m\\u001b[30mw\\u001b[1m\\u001b[30me\\u001b[0m\\u001b[39m\"]\n[59.415563, \"o\", \"\\b\\u001b[1m\\u001b[30me\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[59.492388, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mc\\u001b[0m\\u001b[39m\"]\n[59.563192, \"o\", \"\\b\\u001b[1m\\u001b[30mc\\u001b[1m\\u001b[30ma\\u001b[0m\\u001b[39m\"]\n[59.669657, \"o\", \"\\b\\u001b[1m\\u001b[30ma\\u001b[1m\\u001b[30mn\\u001b[0m\\u001b[39m\"]\n[59.788831, \"o\", \"\\b\\u001b[1m\\u001b[30mn\\u001b[1m\\u001b[30mt\\u001b[0m\\u001b[39m\"]\n[60.30644, \"o\", \"\\b\\u001b[1m\\u001b[30mt\\u001b[1m\\u001b[30m'\\u001b[0m\\u001b[39m\"]\n[60.893462, \"o\", \"\\b\\b\\u001b[1m\\u001b[30mt\\u001b[0m\\u001b[39m\\u001b[0m\\u001b[39m \\b\"]\n[61.020872, \"o\", \"\\b\\b\\u001b[1m\\u001b[30mn\\u001b[0m\\u001b[39m\\u001b[0m\\u001b[39m \\b\"]\n[61.203948, \"o\", \"\\b\\u001b[1m\\u001b[30mn\\u001b[1m\\u001b[30m'\\u001b[0m\\u001b[39m\"]\n[61.323915, \"o\", \"\\b\\u001b[1m\\u001b[30m'\\u001b[1m\\u001b[30mt\\u001b[0m\\u001b[39m\"]\n[61.506352, \"o\", \"\\b\\u001b[1m\\u001b[30mt\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[61.700781, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mr\\u001b[0m\\u001b[39m\"]\n[61.780133, \"o\", \"\\b\\u001b[1m\\u001b[30mr\\u001b[1m\\u001b[30me\\u001b[0m\\u001b[39m\"]\n[61.925858, \"o\", \"\\b\\u001b[1m\\u001b[30me\\u001b[1m\\u001b[30mt\\u001b[0m\\u001b[39m\"]\n[62.05761, \"o\", \"\\b\\u001b[1m\\u001b[30mt\\u001b[1m\\u001b[30mr\\u001b[0m\\u001b[39m\"]\n[62.198445, \"o\", \"\\b\\u001b[1m\\u001b[30mr\\u001b[1m\\u001b[30mi\\u001b[0m\\u001b[39m\"]\n[62.30557, \"o\", \"\\b\\u001b[1m\\u001b[30mi\\u001b[1m\\u001b[30me\\u001b[0m\\u001b[39m\"]\n[62.443085, \"o\", \"\\b\\u001b[1m\\u001b[30me\\u001b[1m\\u001b[30mv\\u001b[0m\\u001b[39m\"]\n[62.51056, \"o\", \"\\b\\u001b[1m\\u001b[30mv\\u001b[1m\\u001b[30me\\u001b[0m\\u001b[39m\"]\n[62.596494, \"o\", \"\\b\\u001b[1m\\u001b[30me\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[62.92443, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mi\\u001b[0m\\u001b[39m\"]\n[63.095964, \"o\", \"\\b\\u001b[1m\\u001b[30mi\\u001b[1m\\u001b[30mt\\u001b[0m\\u001b[39m\"]\n[63.199589, \"o\", \"\\b\\u001b[1m\\u001b[30mt\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[63.296012, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mf\\u001b[0m\\u001b[39m\"]\n[63.435393, \"o\", \"\\b\\u001b[1m\\u001b[30mf\\u001b[1m\\u001b[30mr\\u001b[0m\\u001b[39m\"]\n[63.478495, \"o\", \"\\b\\u001b[1m\\u001b[30mr\\u001b[1m\\u001b[30mo\\u001b[0m\\u001b[39m\"]\n[63.635312, \"o\", \"\\b\\u001b[1m\\u001b[30mo\\u001b[1m\\u001b[30mm\\u001b[0m\\u001b[39m\"]\n[63.73524, \"o\", \"\\b\\u001b[1m\\u001b[30mm\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[64.245442, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mn\\u001b[0m\\u001b[39m\"]\n[64.323737, \"o\", \"\\b\\u001b[1m\\u001b[30mn\\u001b[1m\\u001b[30me\\u001b[0m\\u001b[39m\"]\n[64.42844, \"o\", \"\\b\\u001b[1m\\u001b[30me\\u001b[1m\\u001b[30mt\\u001b[0m\\u001b[39m\"]\n[64.573704, \"o\", \"\\b\\u001b[1m\\u001b[30mt\\u001b[1m\\u001b[30mw\\u001b[0m\\u001b[39m\"]\n[64.641164, \"o\", \"\\b\\u001b[1m\\u001b[30mw\\u001b[1m\\u001b[30mo\\u001b[0m\\u001b[39m\"]\n[64.775938, \"o\", \"\\b\\u001b[1m\\u001b[30mo\\u001b[1m\\u001b[30mr\\u001b[0m\\u001b[39m\"]\n[64.824792, \"o\", \"\\b\\u001b[1m\\u001b[30mr\\u001b[1m\\u001b[30mk\\u001b[0m\\u001b[39m\"]\n[65.100005, \"o\", \"\\b\\u001b[1m\\u001b[30mk\\u001b[1m\\u001b[30m.\\u001b[0m\\u001b[39m\"]\n[65.324098, \"o\", \"\\u001b[?1l\\u001b>\"]\n[65.328114, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[65.330857, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[65.331904, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig\\u0007\\u001b]1;~/dev/brig\\u0007\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[65.332884, \"o\", \"\\u001b[?1h\\u001b=\"]\n[65.33457, \"o\", \"\\u001b[?2004h\"]\n[66.221421, \"o\", \"\\u001b[1m\\u001b[30m#\\u001b[0m\\u001b[39m\"]\n[66.339489, \"o\", \"\\b\\u001b[1m\\u001b[30m#\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[66.443802, \"o\", \"\\b\\b\\u001b[1m\\u001b[30m#\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30ml\\u001b[0m\\u001b[39m\"]\n[66.602202, \"o\", \"\\b\\u001b[1m\\u001b[30ml\\u001b[1m\\u001b[30mi\\u001b[0m\\u001b[39m\"]\n[67.462451, \"o\", \"\\b\\u001b[1m\\u001b[30mi\\u001b[1m\\u001b[30mk\\u001b[0m\\u001b[39m\"]\n[67.57586, \"o\", \"\\b\\u001b[1m\\u001b[30mk\\u001b[1m\\u001b[30me\\u001b[0m\\u001b[39m\"]\n[67.659472, \"o\", \"\\b\\u001b[1m\\u001b[30me\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[67.920771, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mw\\u001b[0m\\u001b[39m\"]\n[68.070952, \"o\", \"\\b\\u001b[1m\\u001b[30mw\\u001b[1m\\u001b[30me\\u001b[0m\\u001b[39m\"]\n[68.15706, \"o\", \"\\b\\u001b[1m\\u001b[30me\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[68.239042, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30md\\u001b[0m\\u001b[39m\"]\n[68.330446, \"o\", \"\\b\\u001b[1m\\u001b[30md\\u001b[1m\\u001b[30mi\\u001b[0m\\u001b[39m\"]\n[68.423091, \"o\", \"\\b\\u001b[1m\\u001b[30mi\\u001b[1m\\u001b[30md\\u001b[0m\\u001b[39m\"]\n[68.538495, \"o\", \"\\b\\u001b[1m\\u001b[30md\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[68.656006, \"o\", \"\\b\\u001b[1m\\u001b[30m \\u001b[1m\\u001b[30mb\\u001b[0m\\u001b[39m\"]\n[68.740476, \"o\", \"\\b\\u001b[1m\\u001b[30mb\\u001b[1m\\u001b[30me\\u001b[0m\\u001b[39m\"]\n[68.869553, \"o\", \"\\b\\u001b[1m\\u001b[30me\\u001b[1m\\u001b[30mf\\u001b[0m\\u001b[39m\"]\n[69.035796, \"o\", \"\\b\\u001b[1m\\u001b[30mf\\u001b[1m\\u001b[30mo\\u001b[0m\\u001b[39m\"]\n[69.169299, \"o\", \"\\b\\u001b[1m\\u001b[30mo\\u001b[1m\\u001b[30mr\\u001b[0m\\u001b[39m\"]\n[69.244378, \"o\", \"\\b\\u001b[1m\\u001b[30mr\\u001b[1m\\u001b[30me\\u001b[0m\\u001b[39m\"]\n[69.311449, \"o\", \"\\b\\u001b[1m\\u001b[30me\\u001b[1m\\u001b[30m.\\u001b[0m\\u001b[39m\"]\n[69.504344, \"o\", \"\\b\\u001b[1m\\u001b[30m.\\u001b[1m\\u001b[30m \\u001b[0m\\u001b[39m\"]\n[69.947017, \"o\", \"\\b\\b\\u001b[1m\\u001b[30m.\\u001b[0m\\u001b[39m\\u001b[0m\\u001b[39m \\b\"]\n[70.356133, \"o\", \"\\u001b[?1l\\u001b>\"]\n[70.358766, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n[70.361407, \"o\", \"\\u001b[1m\\u001b[7m%\\u001b[27m\\u001b[1m\\u001b[0m \\r \\r\"]\n[70.361888, \"o\", \"\\u001b]2;sahib@werkbank: ~/dev/brig\\u0007\\u001b]1;~/dev/brig\\u0007\"]\n[70.362171, \"o\", \"\\r\\u001b[0m\\u001b[27m\\u001b[24m\\u001b[Jλ \\u001b[K\"]\n[70.362785, \"o\", \"\\u001b[?1h\\u001b=\"]\n[70.365401, \"o\", \"\\u001b[?2004h\"]\n[71.101157, \"o\", \"\\u001b[?2004l\\r\\r\\n\"]\n" }, { "path": "docs/conf.py", "content": "#!/usr/bin/env python3\n# -*- coding: utf-8 -*-\n#\n# brig documentation build configuration file, created by\n# sphinx-quickstart on Sun Dec 24 00:44:21 2017.\n#\n# This file is execfile()d with the current directory set to its\n# containing dir.\n#\n# Note that not all possible configuration values are present in this\n# autogenerated file.\n#\n# All configuration values have a default; values that are commented out\n# serve to show the default.\n\n# If extensions (or modules to document with autodoc) are in another directory,\n# add these directories to sys.path here. If the directory is relative to the\n# documentation root, use os.path.abspath to make it absolute, like shown here.\n#\nimport os\nimport subprocess\n# import sys\n# sys.path.insert(0, os.path.abspath('.'))\n\n# -- General configuration ------------------------------------------------\n\n# If your documentation needs a minimal Sphinx version, state it here.\n#\n# needs_sphinx = '1.0'\n\n# Add any Sphinx extension module names here, as strings. They can be\n# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom\n# ones.\nextensions = [\n 'sphinx.ext.mathjax',\n 'sphinx.ext.todo',\n 'sphinx.ext.ifconfig',\n 'sphinx.ext.githubpages',\n 'sphinxcontrib.fulltoc',\n]\n\n\n\n# Add any paths that contain templates here, relative to this directory.\ntemplates_path = ['_templates']\n\n# The suffix(es) of source filenames.\n# You can specify multiple suffix as a list of string:\n#\n# source_suffix = ['.rst', '.md']\nsource_suffix = '.rst'\n\n# The master toctree document.\nmaster_doc = 'index'\n\n# General information about the project.\nproject = 'brig'\ncopyright = '2016-2020, Chris Pahl'\nauthor = 'Chris Pahl'\n\n# The version info for the project you're documenting, acts as replacement for\n# |version| and |release|, also used in various other places throughout the\n# built documents.\n\n# The short X.Y version.\ndef get_version_from_git():\n try:\n proc = subprocess.run(\n [\"git\", \"tag\", \"--sort\", \"v:refname\"],\n capture_output=True,\n text=True,\n check=True,\n )\n\n tag_list = [tag.strip() for tag in proc.stdout.split('\\n') if tag]\n return tag_list[-1].strip()\n except Exception as e:\n print(\"-- note: could not read version from git: {}\".format(e))\n return 'v0.0.0'\n\n\n# The full version, including alpha/beta/rc tags.\nrelease = get_version_from_git()\nversion = release\n\n# The language for content autogenerated by Sphinx. Refer to documentation\n# for a list of supported languages.\n#\n# This is also used if you do content translation via gettext catalogs.\n# Usually you set \"language\" from the command line for these cases.\nlanguage = None\n\n# List of patterns, relative to source directory, that match files and\n# directories to ignore when looking for source files.\n# This patterns also effect to html_static_path and html_extra_path\nexclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', 'talk/*.rst']\n\n# The name of the Pygments (syntax highlighting) style to use.\npygments_style = 'sphinx'\n\n# If true, `todo` and `todoList` produce output, else they produce nothing.\ntodo_include_todos = True\n\n# -- Options for HTML output ----------------------------------------------\n\n# The theme to use for HTML and HTML Help pages. See the documentation for\n# a list of builtin themes.\n# html_theme = 'sphinx_rtd_theme'\n\ntry:\n import sphinx_bootstrap_theme\n\n html_theme = 'bootstrap'\n html_theme_path = sphinx_bootstrap_theme.get_html_theme_path()\nexcept ImportError:\n print(\"-- no »sphinx_bootstrap_theme« is installed.\")\n print(\"-- install it via requirements.txt, if you want it.\")\n print(\"-- falling back to default theme.\")\n html_theme = 'sphinx_rtd_theme'\n\n# Theme options are theme-specific and customize the look and feel of a theme\n# further. For a list of options available for each theme, see the\n# documentation.\nhtml_theme_options = {\n # Navigation bar title. (Default: ``project`` value)\n 'navbar_title': \"brig\",\n\n # Tab name for entire site. (Default: \"Site\")\n 'navbar_site_name': \"Table of Contents\",\n\n # A list of tuples containing pages or urls to link to.\n # Valid tuples should be in the following forms:\n # (name, page) # a link to a page\n # (name, \"/aa/bb\", 1) # a link to an arbitrary relative url\n # (name, \"http://example.com\", True) # arbitrary absolute url\n # Note the \"1\" or \"True\" value above as the third argument to indicate\n # an arbitrary url.\n 'navbar_links': [\n (\"GitHub\", \"https://github.com/sahib/brig\", True),\n (\"Build Status\", \"https://travis-ci.org/sahib/brig\", True),\n ],\n\n # Render the next and previous page links in navbar. (Default: true)\n 'navbar_sidebarrel': False,\n\n # Render the current pages TOC in the navbar. (Default: true)\n 'navbar_pagenav': False,\n\n # Tab name for the current pages TOC. (Default: \"Page\")\n 'navbar_pagenav_name': \"Page\",\n\n # Global TOC depth for \"site\" navbar tab. (Default: 1)\n # Switching to -1 shows all levels.\n 'globaltoc_depth': 2,\n\n # Include hidden TOCs in Site navbar?\n #\n # Note: If this is \"false\", you cannot have mixed ``:hidden:`` and\n # non-hidden ``toctree`` directives in the same page, or else the build\n # will break.\n #\n # Values: \"true\" (default) or \"false\"\n 'globaltoc_includehidden': \"true\",\n\n # HTML navbar class (Default: \"navbar\") to attach to
element.\n # For black navbar, do \"navbar navbar-inverse\"\n 'navbar_class': \"navbar\",\n\n # Fix navigation bar to top of page?\n # Values: \"true\" (default) or \"false\"\n 'navbar_fixed_top': \"true\",\n\n # Location of link to source.\n # Options are \"nav\" (default), \"footer\" or anything else to exclude.\n 'source_link_position': \"footer\",\n\n # Bootswatch (http://bootswatch.com/) theme.\n #\n # Options are nothing (default) or the name of a valid theme\n # such as \"cosmo\" or \"sandstone\".\n #\n # The set of valid themes depend on the version of Bootstrap\n # that's used (the next config option).\n #\n # Currently, the supported themes are:\n # - Bootstrap 2: https://bootswatch.com/2\n # - Bootstrap 3: https://bootswatch.com/3\n 'bootswatch_theme': \"flatly\",\n\n # Choose Bootstrap version.\n # Values: \"3\" (default) or \"2\" (in quotes)\n 'bootstrap_version': \"3\",\n}\n\n# Add any paths that contain custom static files (such as style sheets) here,\n# relative to this directory. They are copied after the builtin static files,\n# so a file named \"default.css\" will overwrite the builtin \"default.css\".\nhtml_static_path = ['_static']\n\n# Custom sidebar templates, must be a dictionary that maps document names\n# to template names.\n#\n# This is required for the alabaster theme\n# refs: http://alabaster.readthedocs.io/en/latest/installation.html#sidebars\nhtml_sidebars = {\n 'tutorial/*': [\n 'localtoc.html',\n ],\n 'quickstart*': [\n 'localtoc.html',\n ],\n 'installation*': [\n 'localtoc.html',\n ],\n 'faq*': [\n 'localtoc.html',\n ],\n 'roadmap*': [\n 'localtoc.html',\n ],\n 'feature*': [\n 'localtoc.html',\n ],\n 'contributing*': [\n 'localtoc.html',\n ]\n}\n\n\n# -- Options for HTMLHelp output ------------------------------------------\n\n# Output file base name for HTML help builder.\nhtmlhelp_basename = 'brigdoc'\n\n# # -- Options for LaTeX output ---------------------------------------------\n# \n# latex_elements = {\n# # The paper size ('letterpaper' or 'a4paper').\n# #\n# # 'papersize': 'letterpaper',\n# \n# # The font size ('10pt', '11pt' or '12pt').\n# #\n# # 'pointsize': '10pt',\n# \n# # Additional stuff for the LaTeX preamble.\n# #\n# # 'preamble': '',\n# \n# # Latex figure (float) alignment\n# #\n# # 'figure_align': 'htbp',\n# }\n# \n# # Grouping the document tree into LaTeX files. List of tuples\n# # (source start file, target name, title,\n# # author, documentclass [howto, manual, or own class]).\n# latex_documents = [\n# (master_doc, 'brig.tex', 'brig Documentation',\n# 'Chris Pahl', 'manual'),\n# ]\n# \n# \n# # -- Options for manual page output ---------------------------------------\n# \n# # One entry per manual page. List of tuples\n# # (source start file, name, description, authors, manual section).\n# man_pages = [\n# (master_doc, 'brig', 'brig Documentation',\n# [author], 1)\n# ]\n# \n# \n# # -- Options for Texinfo output -------------------------------------------\n# \n# # Grouping the document tree into Texinfo files. List of tuples\n# # (source start file, target name, title, author,\n# # dir menu entry, description, category)\n# texinfo_documents = [\n# (master_doc, 'brig', 'brig Documentation',\n# author, 'brig', 'One line description of project.',\n# 'Miscellaneous'),\n# ]\n\n\ndef setup(app):\n # app.add_stylesheet('css/custom.css')\n app.add_css_file('css/custom.css')\n" }, { "path": "docs/contributing.rst", "content": "How to contribute\n=================\n\nSomething we would be especially interested in are *experience reports:* We\nwant you to try out the current state of the software and write down the\nfollowing:\n\n- Was it easy to get »brig« running?\n- Was it easy to understand its concepts?\n- What is your intended usecase for it? Could you make it work?\n- If no, what's missing in your opinion to make the usecase possible?\n- Anything else that you feel free to share.\n\nThose reports should be posted as GitHub issue. They will help us to develop\nbrig further in the \"bigger picture\".\n\nAlso, the developer of this software is currently doing all of this is in his\nfree time. If you're willing to offer any financial support feel free to\ncontact me.\n\nWhat to improve\n---------------\n\nWe try to open a ticket for anything that can be worked right now\n\nThe following general improvements are of course also greatly appreciated:\n\n- Bug reports & fixes.\n- Documentation improvements.\n- Writing more and better tests.\n- Porting to other platforms.\n\nWorkflow\n--------\n\nPlease adhere to the general `GitHub workflow_`, i.e. fork the repository,\nmake your changes and open a pull request that can be discussed.\n\n.. _`Github workflow`: https://help.github.com/articles/about-pull-requests\n\nHere's a small checklist before publishing your pull request:\n\n* Did you ``go fmt`` all code?\n* Does your code style fit with the rest of the code base?\n* Did you run ``task lint``?\n* Did you write tests if necessary?\n* Did you consider if changes to the docs are necessary?\n* Did you check if you need something to CHANGELOG.md?\n\nThank you for your contribution.\n" }, { "path": "docs/faq.rst", "content": "Frequently Asked Questions\n==========================\n\nGeneral questions\n-----------------\n\n1. Why is the software named ``brig``?\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nIt is named after the ship with the same name.\nWhen we named it, we thought it's a good name for the following reason:\n\n- A ``brig`` is a very lightweight and fast ship.\n- It was commonly used to transport small amount of goods.\n- A ship operates on streams (sorry 😛)\n- The name is short and somewhat similar to ``git``.\n- It gives you a few nautical metaphors and a logo for free.\n- Words like »bright«, »brigade« and many others start with it\n\nTruth be told, only half of the two name givers thought it's a good name, but\nI still kinda like it.\n\n2. Who develops it?\n~~~~~~~~~~~~~~~~~~~\n\nAlthough this documentation sometimes speaks of »we«, the only developer is\ncurrently `Chris Pahl `_. He writes it entirely in\nhis free time, mostly during commuting with the train.\n\nTechnical questions\n-------------------\n\n1. How is the encryption working?\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nA stream is chunked into equal sized blocks that are encrypted in GCM mode\nusing AES-256. Additionally ChaCha20 (with Poly1305) is currently supported but\nit might be removed soon. The overall file format is somewhat similar to NaCL\nsecretboxes, but it is more tailored to supporting efficient seeking.\n\nThe current default is ``ChaCha20``, although machines with the ``aes-ni``\ninstruction set might yield significant higher throughput. The source of the\n`encryption layer can be found here `_.\nHere's a basic overview over the format:\n\n.. image:: _static/format-encryption.svg\n :width: 66%\n :align: center\n\nThe key of each file is currently being derived from the content hash of the\nfile (See also `Convergent Encryption\n`_). If the content\nchanges later, the key does not change since the key is only generated once\nduring the first staging of the file.\n\nPlease refer to the implementation for all implementation details for now. No\nsecurity audits of the implementation have been done yet, therefore I'd\nappreciate every pair of eyes. Especially while everything is still in flux and\nwon't harm any users.\n\n2. Is there compression implemented?\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nYes. The compression is being done before encryption and is only enabled if the\nfile looks compression-worthy. The »worthiness« is determined by looking at its\nheader to guess a mime-type. Depending on the mime-type either ``snappy`` or\n``lz4`` is selected or no compression is added at all.\n\nThe source of the `compression layer can be found here `_. Here's\na basic overview over the format:\n\n.. image:: _static/format-compression.svg\n :width: 66%\n :align: center\n\n3. What hash algorithms are used?\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nTwo algorithms are used:\n\n* ``SHA256`` is used by ``IPFS`` for every backend hash.\n* ``SHA3-256`` is used as general purpose hash for everything ``brig`` internal\n (Content and Tree hash).\n\nEach hash is encoded as `multihash\n`_. For output purposes this\nrepresentation is encoded additionally in ``base58``. Therefore, all hashes\nthat start with ``W1`` are ``sha3-256`` hashes while the ones starting with\n``Qm`` are ``sha256`` hashes. Keep in mind that ``base58`` is case-sensitive.\n\n4. What kind of deduplication is currently used?\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nIt is currently only possible to deduplicate between individual versions of a file.\nAnd there also only the portion before the modification.\n\n``IPFS`` implements deduplication, but it is circumvented by encrypting blocks\nbefore giving them over to the backend. Implementing a more proper and informed\ndeduplication is one of the long term goals, which require more thorough\ninteraction with ``IPFS``. It is also possible to do some basic deduplication\npurely on ``brig`` side since we have more info on the file than ``IPFS`` has.\n\n5. How fast is the I/O when using ``brig``?\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nHere are some rather outdated graphs where you can get a rough feeling how fast\nit can be. There are a few rules of thumb with mostly obvious content:\n\n* It it goes over the network, it's the network speed plus a smaller constant overhead.\n* If it comes over FUSE, it is quite a bit slower than over ``brig cat``.\n* If you do not use compression, writing and reading will be faster.\n\nThe graphs below only measure in-memory performance compared to a ``dd`` like\nspeed (see the »baseline« line).\n\n.. image:: _static/movie_read.svg\n :width: 66%\n\n.. image:: _static/movie_write.svg\n :width: 66%\n\nYour mileage may vary and you better do your own benchmarks for now.\n\n.. todo::\n\n Explain/Update those graphs.\n" }, { "path": "docs/features.rst", "content": ".. _features-page:\n\nFeatures\n========\n\n.. note::\n\n The featuers below are actually available, but before version 1.0.0 we\n won't give any guarantees regarding stability or edge cases. Your mileage\n may vary currently.\n\n\nEncrypted and compression built-in\n----------------------------------\n\n* All data is encrypted during storage and transport using AES-256 in GCM mode.\n* Optional compression algorithm is selected based on the file type.\n* Hints can be given to change the default algorithm for certain or all files.\n* Keys are stored as part of the metadata during synchronisation.\n\nEasy Version control\n--------------------\n\n* Simplified ``git``-like version control only limited by your storage space.\n* Synchronization algorithm that can handle `moved files\n `_\n and `empty directories\n `_\n and files.\n* Auto-updating facility that will sync on any change.\n* Configurable conflict handling.\n\nSeparation between data and metadata\n------------------------------------\n\n* Your data does not need to be stored on the device you are currently using.\n* Pin the data you want to use to your local storage. Every repository acts as\n cache of all the files you have access to.\n* Keep a range of versions cached locally and delete older versions if they\n exceed a quota.\n\nTruly Distributed\n-----------------\n\n* No central server at all. All infrastructure is based on IPFS.\n* Still, central architectures can be build with ``brig``.\n* Simple user identification and discovery.\n* You do not store data you don't want to store. Pin what you need,\n fetch the rest from the network on request.\n\nFUSE filesystem\n---------------\n\n* FUSE filesystem mirrors your data to a local directory.\n* Allows your normal tools to work seamlessly with brig.\n* Mounts can be persisted to stay where they are.\n* Not high performance, but fast enough for daily usage.\n\nGateway and Web UI\n------------------\n\n* Gateway to share normal HTTP/S links with other users.\n* Simple UI provided to execute the typical tasks without the need of a command line.\n* User and right management included.\n\n.. image:: _static/gateway-files.png\n :alt: Gateway files view\n :width: 66%\n\n100% Open-Source\n----------------\n\n* Completely free software under the terms of the ``AGPL``.\n* Development driven by the community.\n* Written in Go and Elm.\n\n---------\n\nComparison with other tools\n===========================\n\nWhen showing ``brig`` (or any other software in general) to someone the first\nquestion is usually something like *»But isn't there already X?«* and sometimes\neven *»Why don't you just contribute to other projects?«*. This section tries\nto find an answer to both questions. The answer will obviously be biased, so\ntake it with a fair grain of salt.\n\nYes, there is other software in this world. But this is always a matter of\ntrade offs the author of each individual package has chosen. One application\nmight not run on your platform, the next might not be secure enough for your\nneeds, the other one is proprietary or has something else that does not fit\nyour liking. I won't go into an exhaustive list of competitors, but more\nhighlight the things that are special in ``brig`` and cannot be done easily in\nother systems.\n\nI said »competitors« earlier, which is a silly term, since I don't see this as\na competition. For me it's more about giving the user a choice and improving by\nadapting good ideas from other implementations. Let's list a few of those\n»competitors« to give you an impression about the place of ``brig`` in the\nworld:\n\n* `Syncthing `_: Probably conceptually the nearest\n relative. Also a peer-to-peer based filesystem, but with its own protocol.\n Focus seems to be on ease-of-use and general high quality usability. Does not\n have strong versioning. Excellent tool and battle tested.\n* `Resilio `_: Proprietary solution based on BitTorrent. Seems to focus on\n performance and enterprise level resilience. Being proprietary is a show stopper for me.\n* `Perkeep `_: Not focused on files, but on\n storing personal »objects«. Would be probably more interesting as a backend\n for ``brig``.\n* `Upspin `_: A global name system that glues together\n filesystems and other data storage. Could be also a backend for ``brig`` and\n is not directly targeted to end users.\n* `Bazil `_: Basically ``brig`` minus IPFS. While apparently\n discontinued it seems to have a great deal of common features with ``brig``.\n The same author also maintains the FUSE bindings of FUSE and his writeups\n helped me writing the FUSE implementation of ``brig``. Thank you very much\n for your work `@tv42 `_!\n* `Git LFS `_: The large file storage extension to ``git``.\n Similar to ``brig``'s pinning in the sense that large files are replaced with links\n that will be fetched from a LFS server.\n* `git annex `_: Extension to ``git`` that\n tracks filenames and metadata instead of file content. Has a great deal of\n powerful features but can be a bit intimidating to the end users since it does not\n seem to focus much on usability. Features like the number of minimum copies a file\n must have before you can delete it are still on ``brig``'s roadmap.\n\nThere are probably more. Some of these inspired quite a bit how ``brig`` looks\ntoday. So what are the unique features of ``brig`` that you would not get\neasily with other tools?\n\n* **Pinning:** The fact that not all data needs to be on the same machine as the\n ``brig`` daemon opens up interesting possibilities. Also the ability of repinning\n is something I did not see in other tools.\n* **Strong versioning of big files:** High level versioning that is comparable\n to ``git``, but simplified and meant for whole-file version control (and not\n for individual diffs).\n\nOf course there are drawbacks. Choosing ``brig`` currently means using software\nthat is not in widespread use. It did not go through a security audit. It is by\nfar not as efficient as other tools in all use cases. But many of the current hurdles\nare solvable and it's just a matter of time.\n\nThe best advice I can give you: Try it out and see if it fits your use case. If\nit doesn't I'm happy to hear from you and wish you all the best with another\ntool.\n" }, { "path": "docs/index.rst", "content": "``brig`` - decentralized & secure synchronization\n=================================================\n\n.. image:: _static/logo.png\n :width: 50%\n :align: center\n\nWhat is ``brig``?\n-----------------\n\n``brig`` is a distributed & secure file synchronization tool with version\ncontrol. It is based on IPFS, written in Go and will feel familiar to\n``git`` users. Think of it as a swiss army knife for file synchronization or as\na peer to peer alternative to *Dropbox.*\n\n**Key feature highlights:**\n\n* Encryption of data during storage and transport, plus optional compression on the fly.\n* Simplified ``git`` version control only limited by your storage space.\n* Synchronization algorithm that can handle moved files and empty directories and files.\n* Your data does not need to be stored on the device you are currently using.\n* FUSE filesystem that feels like a normal sync folder.\n* No central server at all. Still, central architectures can be build with ``brig``.\n* Gateway and Web based UI to share normal HTTP/S links with other users.\n* Auto-updating facility that will sync on any change.\n* Completely free software under the terms of the ``AGPL``.\n* ...\n\nPlease refer to the :ref:`features-page` for more details. If you want a visual\nhint how ``brig`` looks on the commandline, refer to the :ref:`quickstart`.\n\nWhat is ``brig`` not?\n---------------------\n\n``brig`` tries to focus on being up conceptually simple, by hiding a lot of\ncomplicated details regarding storage and security. Therefore the end result is\nhopefully easy and pleasant to use, while being secure by default. Since\n``brig`` is a »general purpose« tool for file synchronization it of course\ncannot excel in all areas. It won't replace high performance network file\nsystems and should not be used when you are in need of high throughput - at\nleast not at the moment.\n\nI have questions!\n-----------------\n\nPlease ask in one of those places:\n\n* `GitHub Issue Tracker `_: All things like bug reports or feature requests.\n* The matrix chat room ``#brig`` on ``matrix.org``. Just `pick a client `_ and join the room or click `this link `_ directly.\n\nCurrent Status\n--------------\n\n**This software is in active development and probably not suited for production\nuse yet!** But to get it in a stable state, it is **essential** that people\nplay around with it. Consider this is as an open beta phase. Also don't take\nanything granted for now, everything might change wildly before version ``1.0.0``.\n\nWith that being said, ``brig`` is near a somewhat usable state where you can play around\nwith it quite well. All aforementioned features do work, besides possibly being\na little harder to use than ideally possible. A lot of work is currently going into\nstabilizing the current feature set.\n\nAt this moment ``brig`` is **only tested on Linux**. Porting and testing\nefforts are welcome. Other platforms should be able to compile, but there are\ncurrently not guarantees that it will work.\n\nTable of Contents\n-----------------\n\n.. toctree::\n :maxdepth: 2\n :caption: Installation:\n\n installation.rst\n\n.. toctree::\n :maxdepth: 2\n :caption: User manual\n\n tutorial/intro.rst\n tutorial/init.rst\n tutorial/coreutils.rst\n tutorial/mounts.rst\n tutorial/remotes.rst\n tutorial/vcs.rst\n tutorial/pinning.rst\n tutorial/gateway.rst\n tutorial/config.rst\n\n.. toctree::\n :maxdepth: 2\n :caption: Additional resources\n\n quickstart.rst\n faq.rst\n features.rst\n\n.. toctree::\n :maxdepth: 2\n :caption: Development\n\n roadmap.rst\n contributing.rst\n" }, { "path": "docs/installation.rst", "content": "Installation\n------------\n\nWe provide pre-compiled binaries on every release. ``brig`` comes to your computer\nas a single binary that includes everything you need. See here for the release list:\n\n https://github.com/sahib/brig/releases\n\nJust download the binary for you platform, unpack it and put in somewhere in your\n``$PATH`` (for example ``/usr/local/bin``).\n\nIf you trust us well enough, you can also use this online installer to download\nthe latest stable ``brig`` binary to your current working directory:\n\n.. code-block:: bash\n\n $ bash <(curl -s https://raw.githubusercontent.com/sahib/brig/master/scripts/install.sh)\n\nSpecific distributions\n----------------------\n\nSome distributions can install ``brig`` directly via their package manager.\nThose are currently:\n\n* Arch Linux (`PKGBUILD `_; builds ``develop`` branch)\n\nCompiling yourself\n------------------\n\nIf you use a platform we don't provide binaries for or if you want to use\na development version, you're going have to compile ``brig`` yourself. But\ndon't worry that's quite easy. We do not have many dependencies. You only need\ntwo things: The programming language *Go* and the version control system\n``git``.\n\nStep 0: Installing Go\n~~~~~~~~~~~~~~~~~~~~~\n\nThis is only required if you don't already have ``Go`` installed.\nPlease consult your package manager for that.\n\n.. warning::\n\n ``brig`` only works with a newer version of Go (>= 1.10).\n The version in your package manager might be too outdated,\n if you're on e.g. Debian. Make sure it's rather up to date!\n If it's too old you can always use tools like ``gvm`` to get a more recent version.\n\n\nIf you did not do that, you gonna need to install ``Go``. `Refere here\n`_ for possible ways of doing so. Remember to\nset the ``GOPATH`` environment variable to a place where you'd like to have\nyour ``.go`` sources being placed. For example you can put this in your\n``.bashrc``:\n\n.. code:: bash\n\n # Place the go sources in a \"go\" directory inside your home directory:\n export GOPATH=~/go\n # This is needed for the go toolchain:\n export GOBIN=\"$GOPATH/bin\"\n # Make sure that our shell finds the go binaries:\n export PATH=\"$GOPATH/bin:$PATH\"\n\nBy choosing to have the ``GOPATH`` in your home directory you're not required\nto have ``sudo`` permissions later on. You also need to have ``git``\n`installed `_ for the next step.\n\nStep 1: Compile & Install ``brig``\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nThis step requires setting ``GOPATH``, as discussed in the previous section.\n\n.. code:: bash\n\n $ go get -d -v -u github.com/sahib/brig # Download the sources.\n $ cd $GOPATH/src/github.com/sahib/brig # Go to the source directory.\n $ ./scripts/install-task.sh # Install the build system.\n $ task # Build the binary.\n\nExecution might take a few minutes though because all of ``brig`` is being\ncompiled during the ``task`` step - this also includes the download of all\ndependencies.\n\nIf you cannot or want to install ``git`` for some reason, you can `manually\ndownload a zip `_ from GitHub\nand place its contents into ``$GOPATH/src/github.com/sahib/brig``. In this\ncase, you can skip the ``go get`` step.\n\nStep 2: Test if the installation is working\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nIf everything worked, there will be a ``brig`` binary in ``$GOBIN``.\n\n.. code:: bash\n\n $ brig help\n\nIf above command prints out documentation on how to use the program's\ncommandline switches then the installation worked. Happy file shipping!\n\nSetting up IPFS\n---------------\n\n``brig`` requires a running *IPFS* daemon. While ``brig`` has ways to do install a IPFS daemon for you,\nit is preferable to install it via your package manager or via the official way:\n\n https://docs.ipfs.io/introduction/install\n\n-----\n\nContinue with :ref:`getting_started` or directly go to :ref:`quickstart` if you\njust need a refresh on the details.\n" }, { "path": "docs/make.bat", "content": "@ECHO OFF\r\n\r\npushd %~dp0\r\n\r\nREM Command file for Sphinx documentation\r\n\r\nif \"%SPHINXBUILD%\" == \"\" (\r\n\tset SPHINXBUILD=sphinx-build\r\n)\r\nset SOURCEDIR=.\r\nset BUILDDIR=_build\r\nset SPHINXPROJ=brig\r\n\r\nif \"%1\" == \"\" goto help\r\n\r\n%SPHINXBUILD% >NUL 2>NUL\r\nif errorlevel 9009 (\r\n\techo.\r\n\techo.The 'sphinx-build' command was not found. Make sure you have Sphinx\r\n\techo.installed, then set the SPHINXBUILD environment variable to point\r\n\techo.to the full path of the 'sphinx-build' executable. Alternatively you\r\n\techo.may add the Sphinx directory to PATH.\r\n\techo.\r\n\techo.If you don't have Sphinx installed, grab it from\r\n\techo.http://sphinx-doc.org/\r\n\texit /b 1\r\n)\r\n\r\n%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%\r\ngoto end\r\n\r\n:help\r\n%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%\r\n\r\n:end\r\npopd\r\n" }, { "path": "docs/quickstart.rst", "content": ".. warning::\n\n The examples below are slightly outdated and will be revisited at some point.\n All commands should still work, but the output might be a little different now.\n Please refer to the :ref:`getting_started` guide for a more up-to-date version.\n\n.. _quickstart:\n\nQuickstart\n==========\n\nThis does not really explain the philosophy behind ``brig``, but gives a good\nidea what the tool is able to do and how it's supposed to be used. Users\nfamiliar to ``git`` should be able to grok most of the commands intuitively.\n\n1. Init\n-------\n\nBefore you can do anything with ``brig`` you need to create a repository.\nDuring this step, also your online identity will be created. So make sure to\nuse a sane username (``sahib@wald.de``) and resource (``laptop``).\n\n.. raw:: html\n\n \n\n\nAs an alternative to entering your password manually, you can use an existing password manager:\n\n.. raw:: html\n\n \n\n2. Adding files\n---------------\n\nBefore synchronizing them, you need to *stage* them. The files will be stored\nencrypted (and possibly compressed) in blobs on your hard disks.\n\n.. raw:: html\n\n \n\n\n3. Coreutils\n------------\n\n``brig`` provides implementations of most file related core utils like ``mv``,\n``cp``, ``rm``, ``mkdir`` or ``cat``. Handling of files should thus feel\nfamiliar for users that know the command line.\n\n.. raw:: html\n\n \n\n4. Mounting\n-----------\n\nFor daily use and for use with other tools you might prefer a folder that\ncontains the file you gave to ``brig``. This can be done via the built-in FUSE\nlayer.\n\n.. raw:: html\n\n \n\n.. note::\n\n Some built-in commands provided by brig are faster.\n ``brig cp`` for example only copies metadata, while the real ``cp`` will copy the whole file.\n\nIf you wish to always have the mount when ``brig`` is running, you should look\ninto :ref:`permanent-mounts`.\n\n5. Commits\n----------\n\nIn it's heart, ``brig`` is very similar to ``git`` and also supports versioning\nvia commits. In contrast to ``git`` however, there are no branches and you\ncan't go back in history -- you can only bring the history back up front.\n\n.. raw:: html\n\n \n\n6. History\n----------\n\nEach file (and directory) maintains a history of the operations that were done\nto this file.\n\n\n.. raw:: html\n\n \n\n7. Discovery & Remotes\n----------------------\n\nIn order to sync with your buddies, you need to add their *fingerprint* as remotes.\nHow do you get their fingerprint? In the best case by using a separate side channel\nlike telephone, encrypted email or elsewhise. But ``brig`` can assist finding remotes\nvia the ``brig net locate`` command.\n\n.. raw:: html\n\n \n\n.. note::\n\n You should **always** verify the fingerprint is really the one of your buddy.\n ``brig`` cannot do this for you.\n\n8. Sync & Diff\n--------------\n\nOnce both parties have setup each other as remotes, we can easily view and sync\nwith their data.\n\n.. raw:: html\n\n \n\n9. Pinning\n----------\n\nBy default ``brig`` will only keep the most recent files. All other files will\nbe marked to deletions after a certain timeframe. This is done via *Pins*. If\na file is pinned, it won't get deleted. If you don't need a file in local\nstorage, you can also unpin it. On the next access ``brig`` will try to load it\nagain from a peer that provides it (if possible).\n\n.. raw:: html\n\n \n" }, { "path": "docs/requirements.txt", "content": "sphinx_bootstrap_theme==0.6.5\nsphinxcontrib-fulltoc==1.2.0\n" }, { "path": "docs/roadmap.rst", "content": "Roadmap\n=======\n\nThis document lists the improvements that can be done to ``brig`` and (if\npossible) when. All features below are not guaranteed to be implemented and,\ncan be seen more as possible improvements that might change during\nimplementation. Also it should be noted that each future is only an idea and\nnot a fleshed ou implementation plan.\n\nBug fixes and minor changes in handling are not included since this document is\nonly for »big picture« ideas. Also excluded are stability/performance\nimprovements, documentation and testing work, since this is part of the\n»normal« development.\n\nCurrent state\n-------------\n\nThe first real release (0.3.0 »Galloping Galapagos«) was released on the 7th December 2018.\nIt includes all basic features and is working somewhat. The original goals were met:\n\n- Stable command line interface.\n- Git-like version control\n- User discovery\n- User authentication\n- Fuse filesystem\n\nFor day-to-day use there are quite some other features that make brig easier to use\nand capable of forming a Dropbox-like backend out of several nodes.\n\n**There will be no stability guarantees before version 1.0.0.**\n\nFuture\n------\n\nThose features should be considered after releasing the first prototype.\nA certain amount of first user input should be collected to see if the\ndirection we're going is valid.\n\n .. role:: strikethrough\n\n* **Gateway:** :strikethrough:`Provide a built-in (and optional) http server, that can »bridge«\n between the internal ipfs network and people that use a regular browser.\n Instances that run on a public server can then provide hyperlinks of files to\n non-brig users.` *Done as of version 0.3.0.*\n\n* **Config profiles:** Make it easy to configure brig in a way to serve either as thin client\n or as archival node. Archival nodes can be used in cases where a brig network spans over computers\n that lie in a different timezone. The archival node would accumulate all changes and repositories\n would see it as some sort of \"blessed repository\" which holds the latest and greatest state.\n\n* **Automatic syncing:** :strikethrough:`Automatically publish changes after a short amount of time.\n If an instance modified some file other nodes are notified and can decide to\n pull the change.` *Done as of version 0.4.0.*\n\n* **Intelligent pinning strategies:** :strikethrough:`By default only the most recent layer of\n files are being kept. This is very basic and can't be configured currently.\n Some users might only want to have only the last few used files pinned, archive\n instances might want to pin almost everything up to a certain depth.` *Done as of version 0.4.0 (see repinning)*\n\n* *Improve read/write performance:* Big files are currently hold in memory\n completely by the fuse layer (when doing a flush). This is suboptimal and needs\n more intelligent handling and out-of-memory caching of writes. Also, the\n network performance is often very low and ridden by network errors and\n timeouts. This can be tackled since IPFS v0.4.19 supports an --offline switch to\n error out early if a file is not available locally.\n\n* *More automated authentication scheme:* E-Mail-like usernames could be used to\n verify a user without exchanging fingerprints. This could be done by e.g.\n sending an activation code to the email of an user (assuming the brig name is\n the same as his email), which the brig daemon on his side could read and send back.\n\n* *Format and read fingerprint from QR-Code:* Fingerprints are hard to read and\n not so easy to transfer and verify. QR-Code could be a solution here, since we\n could easily snap a picture with a phone camera or print it on a business card.\n\nFar Future\n----------\n\nThose features are also important, but require some more in-depth research or\nmore work and are not the highest priority currently.\n\n* **Port to other platforms:** Especially Windows and eventually Android. This\n relies on external help, since I'm neither capable of porting it, nor really\n a fan of both operating systems.\n\n* **Implement alternative to FUSE:** FUSE currently only works on Linux and is\n therefore not usable outside of that. Windows has something similar (called\n Dokan_). Alternatively we could also go on by implementing a WebDAV server,\n which can also be mounted.\n\n* **Implement the encryption in IPFS:** Having the encryption/compression layer\n in brig effectively disables the usage of deduplication. This is unfortunate\n and could be mitigated by either implementing deduplication ourselves or moving\n to a block based encryption scheme.\n\n.. _dokan: https://github.com/keybase/kbfs/tree/master/dokan\n\n* **Ensure N-Copies:** It should be possible to define a minimum amount of copies\n a file has to have on different peers. This could be maybe incorporated into\n the pinning concept. If a user wants to remove a file, brig should warn him if\n he would violate the min-copies rule. This idea is shamelessly stolen from\n ``git-annex``.\n" }, { "path": "docs/talk/Makefile", "content": "all:\n\thovercraft -N -s index.rst\n" }, { "path": "docs/talk/demo.rst", "content": "0. Preparation\n==============\n\n- Windows: Chrome Incognito (slides, presenter console), Monitor Settings, Terminal (docker, ipfs, hovercraft), Terminal (empty)\n\n-----\n\n- setxkbmap us && xmodmap ~/.xmodmaprc\n- check sound.\n- Check that docker is running.\n- Check that no other brig instance is up.\n- Check: /tmp/{repo,mount} is empty.\n- Do a \"bob-brig ls and bob-brig rmt ls\" to do some pre-caching.\n- Source autocompletion.\n\n1. Init\n=======\n\nUsage is very close to ``git``.\n\n.. code-block:: bash\n\n $ mkdir repo && cd repo\n # Create a new repository in here:\n # Command started einen daemon im Hintergrund!\n $ brig init alice@wonderland.de/laptop\n # Anschaut was brig so angestellt hat:\n $ ls\n # Dann schauen wir mal ob man die Datei ausgeben kann:\n $ brig cat README.md\n\n2. Adding files\n===============\n\n.. code-block:: bash\n\n $ brig stage ~/music.mp3\n $ brig ls\n # Pfadnamen, virtueller root.\n $ brig tree\n $ brig cat music.mp3 | mpv -\n\n\n3. Coreutils\n============\n\n.. code-block:: bash\n\n $ brig mkdir sub\n $ brig cp music.mp3 sub\n $ brig tree\n\n # ähnlich zu `stat` unter linux:\n $ brig info README.md\n $ brig edti README.md\n $ brig edit README.md\n # Hash hat sich nach Edit-Vorgang geändert:\n $ brig info README.md\n\n # Man kann sich ansehen was für daten ipfs dann speichert:\n $ ipfs cat \n -> garbled bullshit.\n\n4. Mounting\n===========\n\n.. code-block:: bash\n\n $ mkdir /tmp/mount\n $ ls /tmp/mount # Empty.\n $ brig mount /tmp/mount\n # Ta-da, alle dateien die man sonst so hat sind auch hier vorhanden:\n $ nautilus /tmp/mount\n # Man kann ganz normal dateien editieren:\n $ vi /tmp/mount/new-file\n $ brig ls\n # Noch nicht sehr performant, aber sowas geht schon:\n $ cp ~/rrd.mkv /tmp/mount\n $ mpv /tmp/mount/rrd.mkv\n\n5. Commits\n==========\n\n.. code-block:: bash\n\n $ brig log\n $ brig diff\n $ brig commit -m 'Added darth vader'\n $ brig log\n $ brig edit README.md\n $ brig mv sub/music.mp3 sub/else.mp3\n $ brig diff # Should print mergeable and moved file.\n\n6. History\n==========\n\n(optional)\n\n.. code-block:: bash\n\n # Etwas anders als git: kein diff an sich:\n $ brig history new-file\n $ brig edit new-file\n $ brig commit -m 'edited new-file'\n $ brig reset HEAD^ new-file\n $ brig cat new-file\n\n7. Discovery & Remotes\n======================\n\n.. code-block:: bash\n\n # bob läuft in einem container auf dem gleichen computer:\n $ bob-brig ls\n $ brig whoami\n # Erst ausführen, dauert etwas:\n $ brig net locate bob\n $ brig remote add $(bob-brig whoami -f)\n $ bob-brig remote add $(brig whoami -f)\n $ brig remote ls\n $ brig remote edit\n\n8 Sync & Diff\n=============\n\n.. code-block:: bash\n\n $ brig remote ls\n $ brig diff bob\n $ brig sync bob\n $ brig log\n $ brig ls\n\n9 Pinning\n=========\n\n.. code-block:: bash\n\n $ brig pin rm # geht.\n $ brig gc\n $ brig cat # geht.\n $ \n $ brig gc\n $ brig cat \n ...blocks...\n\n10 Misc\n=======\n\n.. code-block:: bash\n\n $ brig \n $ brig help stage\n $ brig docs\n $ brig bug\n" }, { "path": "docs/talk/index.rst", "content": ":title: brig\n:author: Chris Pahl\n:css: style.css\n:data-transition-duration: 350\n:data-perspective: 5000\n\n.. role:: white-bg\n.. role:: title-logo\n.. role:: strike\n.. role:: donald\n.. role:: github\n.. role:: www\n.. role:: rtd\n.. role:: underline\n.. role:: small\n\n\n----\n\n.. image:: images/logo.png\n\n:title-logo:`»brig«`\n\n:white-bg:`Ein Werkzeug zur sicheren und verteilten`\n:white-bg:`Dateisynchronisation`\n\n.. note::\n\n - Meine schwer versehrten Damen und Herren\n - Dies ist eine Projektvorstellung.\n - Ihr werdet heute zu Versuchskaninchen ausgebildet.\n\n - \"Unverständlichste Folie\"\n - Begriffserklärung des Titels.\n - Unterscheidung: Synchronisieren / Austauschen\n - \"sicher\" ist schwammig\n - \"dezentral\" heißt ohne zentralen Server (wie git)\n - Name: Zweimaster, wendig, leichtgewichtig, verteilt Datenströme.\n\n----\n\nUm was geht's?\n==============\n\n|\n\n* Einführung\n* Das Problem\n* Demo\n* Was hab ich da grad gesehen?\n* Hilfe!\n* :strike:`Applaus!` Fragen?\n\n.. note::\n\n - Demo nimmt ca. 50% Zeit ein, wird also nicht so trocken.\n - Viel Terminal, wenig Bling-Bling. Also sehr technisch orientiert.\n - Viel Stoff für 45 minuten, muss schnell reden, ihr werdet am Ende abgefragt.\n\n Fragen bitte erst gegen Schluss stellen, bei der Demo könnt ihr aber gern\n was fragen wenn ihr was nicht versteht oder seht.\n\n----\n\nWer ist'n das?\n==============\n\n.. note::\n\n - Aus dieser Hochschule.\n - Vollzeit München.\n - Open Source Entwickler (rmlint)\n - Wer mehr über mich wissen will, darf gern nachher fragen.\n - Wer ich bin, ist ja eigentlich unwichtig…\n Darum geht's in dem Vortrag auch nicht.\n\n|\n\n**Chris Pahl.**\n\n|\n|\n\n:small:`Wer mehr über mich wissen will:`\n\n:small:`https://sahib.github.io`\n\n----\n\nEs war einmal…\n==============\n\n|\n\n.. note::\n\n - Dann mal rein ins Thema...\n - Umfrage: Wer benutzt...\n\n * Dropbox oder andere Cloud Storage Anbieter (OneDrive, Google Drive)\n * ownCloud oder nextCloud\n * Syncthing, git-annex, resilio\n * Was selbst gebasteltetes?\n * git\n\n Ihr seht schon: Es gib so einige Tools die man benutzen kann\n und alle haben unterschiedliche Stärken.\n\n Dropbox ist wohl eins der am meisten genutzten Tools.\n\n.. image:: images/dropbox.png\n\n----\n\nDas Problem\n===========\n\n.. note::\n\n - Ihr erwartet jetzt sicherlich, dass ich euch sage was schlecht an Dropbox ist.\n - Erstmal nicht so viel, es ist sehr einfach benutzbar und meist verfügbar -> verbreitet.\n - zentral, unsicher by default (Zusatzsoftware Boxcryptor), us unternehmen, proprietär.\n - Zusammenarbeit über Dropbox (zB an Quelltext) funktioniert nicht wirklich.\n - Dateiaustausch ist eine Art Babel: Jeder benutzt was anderes.\n - Am längsten dauert der Handshake bis man sich auf's Tool geeinigt hat.\n - Siehe comic.\n\n.. image:: images/xkcd-file-transfer.png\n :width: 75%\n\n----\n\nWas ist das Ziel?\n=================\n\n|\n\n.. image:: images/donald.png\n :align: center\n :width: 50%\n\n|\n\n:donald:`MAKE FILE SYNCING GREAT AGAIN!`\n\n.. note::\n\n Und das machen wir ganz offensichtlich indem wir eine Mauer um die Cloud bauen. :)\n It will be tremendous. Great stuff.\n\n----\n\nGeht das auch detaillierter?\n============================\n\nDinge die Dateiaustausch sein :underline:`sollte`:\n\n|\n\n* Einfach.\n* Sicher.\n* Schnell.\n* Versioniert.\n* Dezentral.\n* Frei.\n\n.. note::\n\n Viele Buzzwords. Und viele davon widersprechen sich auch noch.\n Aber wir reden ja von einer idealen Lösung.\n Spruch: \"Ein Tool das alles kann, kann nichts richtig gut\"\n\n * Einfach: User Discovery, FUSE Filesystem, ist kompatibel, nervt nicht.\n * Sicher: Daten sind by default stets verschlüsselt.\n * Schnell: Eigentlich optional, aber Video Streaming ist nett.\n * Versioniert: git junkie, Zusammenarbeit wird möglich, keine revisions filenames mehr.\n * Dezentral: Datenhoheit. Dropbox hinterlässt ein schlechten Nachgeschmack.\n * Frei: Im Sinne von GPL. Sichere Software muss frei sein - andere Gründe zählen natürlich auch.\n\n----\n\nAber, aber…\n===========\n\n**Ja**, es gibt schon einige dezentrale Tools.\n\n.. image:: images/other-tools.png\n :width: 120%\n :class: inline\n\n* Anderer Fokus.\n* Andere Basis.\n* Andere Features.\n* Gleiches Ziel.\n\n:small:`(Siehe: https://brig.readtheodocs.org/comparison.html)`\n\n.. note::\n\n - So Vergleichdiskussion sind müßig und können den ganzen\n Tag dauern, ohne dass am Ende was dabei rauskommt...\n - Resilio ist proprietär, Syncthing fokusiert sich weniger auf Versionierung mehr auf Usability,\n git-annex genau andersrum.\n - Mein Tool macht aber auch einige Dinge anders, die nicht direkt vergleichbar sind.\n\n Jetzt machen wir hier gedanklich mal einen Cut.\n\n-----\n\nIPFS\n====\n\n.. image:: images/ipfs.png\n\n»Inter-Planetary-File-System«\n\n.. note::\n\n - Ist wie beim Trinken: Man braucht eine gute Basis.\n - Interplanetary Filesystem. Das ist wörtlich zu verstehen.\n - Das ganze soll eine Art dezentrale, sichere versionierte Alternative zum\n heutigen Internet werden. Jeder Nutzer ist Server und Client zugleich\n und hat eine eindeutige ID.\n\n----\n\nWas kann das so?\n================\n\n|\n\n.. code-block:: bash\n\n $ echo 'Hallo Augsburg!' | ipfs add\n added QmbLr7bEQkC85EEGEmQk42dLz25VBy2L6iHyZQu\n\n|\n\n.. code-block:: bash\n\n $ ipfs cat QmbLr7bEQkC85EEGEmQk42dLz25VBy2L6iHyZQu\n Hallo Augsburg!\n\n.. note::\n\n Vorteil: Ganz ohne zentralen Server.\n\n Nachteil: Kann bereits zum filesharing benutzt werden, aber nur sehr\n rudiemntär.\n\n|\n\n.. code-block:: bash\n\n $ ipfs id -f '\\n'\n QmeLNNcryy9Ky1dXnfnEPaDQ2KuJ6yafaSRZssjQ83ie84\n\n----\n\n»brig«\n======\n\n.. image:: images/tux.png\n :class: img-tux\n :width: 25%\n\n.. image:: images/gopher.png\n :class: img-gopher\n :width: 33%\n\n* Hash Nanny für ipfs.\n* In ``Go`` geschrieben.\n* **Zielgruppe:** Linux User. Erstmal.\n\n|\n\nEntwicklungsgeschichte:\n-----------------------\n\n* Ende 2015: *Masterprojekt.*\n* Ende 2016: *Pausiert.*\n* Ende 2017: *Hobbyprojekt.*\n* **Erster Beta Release heute!**\n\n.. note::\n\n - Betonung auf Hash Nanny.\n - Das ist das erste \"beta\" release (0.1.0-beta) - WELTPREMIERE!\n - Mit sehr viel Vorsicht benutzen.\n - Alles kann sich auserdem noch ändern.\n - Release early, release often.\n\n----\n\n:class: small-list\n\nKurz gesagt: Fokus\n==================\n\n- Balance zwischen Sicherheit und Usability.\n- Effizienz ist nett, aber kein primärer Fokus.\n- Kompatibilität zu gewohnten Konzepten.\n- Komplexität hinter einfachen Interface.\n\n.. note::\n\n Natürlich kann kein Tool gleichzeitig einfach zu benutzen, sicher und\n effizient sein. Es soll eine Balance zwischen Benutzbarkeit und Sicherheit\n geben - die Effizienz (hat zumindest momentan) eher drunter gelitten.\n\n Siehe Demo.\n\n----\n\nDemo\n----\n\n.. code-block:: bash\n\n $ brig mv raiders twix\n # sonst ändert sich aber nix.\n\n----\n\n:class: small-list\n\nWorkflow\n========\n\n.. note::\n\n - Synchronisieren kleines Ein mal Eins\n\n* Initialer Setup :small:`(nur einmal)`\n* Remotes finden & hinzufügen :small:`(nur einmal)`\n* Diff anzeigen :small:`(optional)`\n* Synchronisieren\n* Konflikte beheben :small:`(eventuell)`\n\n.. image:: images/workflow.png\n :width: 50%\n :class: workflow\n\n----\n\n:data-rotate: 0\n\nDisclaimer: Sicherheit?\n=======================\n\n.. note::\n\n Ich hab ziemlich oft schon das Wort \"sicher\" benutzt.\n Wenn ich sagen würde, dass »brig« sicher ist, dann heißt das eigentlich\n nur dass ich beim Schreiben der Software die Absicht hatte, sichere Software zu schreiben.\n\n Es kommt auf die Angriffsvektoren an.\n Und selbst wenn ich das geschafft hätte, dann kann man das Tool sicher benutzen,\n aber jemand könnte immer noch an deinen ungelockten PC gehen... (uvm)\n\n Übertragung und Speicherung ist sicher gemacht, aber man könnte zb derzeit\n trotzdem mit wenig Mühe herausfinden wer mit wem kommuniziert.\n\n Philosophie ist allgemein: Ein Schloss, dass man nur unter Mühe öffnen\n kann, benutzt kaum einer.\n\n|\n\n.. image:: images/xkcd-security.png\n :width: 110%\n\n----\n\n:data-rotate: 90\n\nDezentralität\n=============\n\n.. note::\n\n - Was heißt jetzt eigentlich dezentral?\n - Problem: Beide müssen zur selben Zeit online sein.\n - Braucht evtl. archiv instanz.\n - funktioniert besser oder schlechter abhaengig vom Usecase:\n - Austausch von Folien und Notizen zwischen Studenten und Professoren: gut.\n - Einseitiges Herunterladen von Formularen bei einer Behoerde: schlecht.\n\n.. image:: images/map.png\n :width: 120%\n\n----\n\n:class: small-list\n\n:data-rotate: 180\n\nNutzermanagement\n================\n\n…existiert nicht.\n\n|\n\n.. note::\n\n - ...Ist nicht wirklich vorhanden.\n - Es gibt keine registrierten Nutzer.\n - Zwei Nutzer können den selben Displaynamen haben!\n - Aber nicht den selben Fingerprint.\n - Email bzw. Jabber ID ähnlich.\n\n|\n\n.. image:: images/id.png\n :width: 100%\n\n----\n\n:data-rotate: 270\n\nVersionierung\n-------------\n\n.. image:: images/mona.png\n :width: 100%\n\n.. note::\n\n - brig = git - diff\n - versionierung hilft im Alltag, aber git ist normal nicht tauglich dafür.\n\n----\n\n:data-rotate: 360\n\nPinning\n=======\n\n.. note::\n\n - Pinning Beispiel: Musik ordner auf Handy und Server.\n - Mechanismus um Sachen wie \"Speichere alle Dateiversionen der 30 letzten Tage\" zu implementieren.\n - Komplette Separation von Daten und Metadaten.\n\n|\n|\n\n.. image:: images/pin.png\n :width: 40%\n\n\n----\n\nRoadmap\n=======\n\n:class: small-list\n\n- Selektives Sharing.\n- Gateway für Hyperlinks.\n- Archive Instances.\n- Autosync und vieles mehr...\n- Performance, mehr Dokumentation...\n\n|\n\n.. image:: images/binocs.png\n :class: future\n :width: 40%\n\n|\n\n**Hauptproblem:** Nur ein Entwickler.\n\n.. note::\n\n ... und der arbeitet nen Vollzeitjob.\n\n Keine gute Basis für eine stabile Weiterentwicklung.\n\n Features die noch kommen sollen:\n\n - Knoten, die automatisch synchroniseren (als »blessed repo« wie bei git)\n - Fingerprints als QR Code\n - Mobile Version mit simplen Dateibrowser.\n - Verbessertes User-Management.\n\n----\n\n:id: help-slide\n\nHilfe? Erwünscht.\n=================\n\n.. note::\n\n Problem: Man macht ein Release und kriegt 20 Feature Requests,\n mit teils total widersprüchlichen Anforderungen.\n Das artet in Feature-itis aus (-> Wollmilchsau)\n\n Am Ende steht man mit eine Software da, die Kaffee kochen kann,\n dafür aber nur so mittel und dessen Name mit \"j\" beginnt.\n (Müsst ihr mal drauf aufpassen... jDownloader, jQuery, java)\n\n Experience Reports:\n\n - Fokus auf Problemen, nicht auf Lösungen.\n - Was ihr tun wolltet\n - Was ihr eigentlich gemacht/erwartet habt\n - Warum das nicht so ganz funktioniert hat\n - Bitte dafür ein issue report auf github aufmachen.\n\nMithilfe via **Experience Reports.**\n\n1. Was wolltet ihr machen?\n2. Was habt ihr versucht?\n3. Warum ging es nicht?\n\n.. image:: images/wollmilchsau.png\n :class: wollmilchsau\n :width: 45%\n\n|\n\nUnd sonst?\n----------\n\n- Bug reports. :small:`(brig bug)`\n- Pull requests mit Fixes.\n- **Keine** Feature Requests!\n\n----\n\nProbem gelöst?\n==============\n\n.. note::\n\n Sagt ihr es mir...\n\n - Ja, die Lösung ist also ganz einfach... man schreibt einfach ein Tool\n das alles richtig macht, jeder nutzt das und gut ist.\n - Abe ja, sagt ihr es mir: Waere so ein Tool hilfreich fuer manche von euch?\n - Zugegeben: Es ist noch nicht so einfach benutzbar dank Terminal, aber das kann\n ja noch werden. Geht ja erstmal um die technische Basis.\n - Die Zeit wird es zeigen.\n\n.. image:: images/xkcd-standards.png\n :width: 110%\n\n----\n\nLetzte Worte\n=============\n\n.. note::\n\n Wer Fragen oder Anmerkungen jetzt abgeben will, der möges dies nun tun oder für immer schweigen.\n Ansonsten könnt ihr mich natürlich auch einfach auf dem Gang ansprechen.\n\n Slides sind online auf der online schedule.\n\n|\n\n:rtd:`http://brig.rtfd.org`\n\n:github:`github.com/sahib/brig`\n\n:www:`http://sahib.github.io/brig/public`\n\n|\n\n*Fragen?*\n" }, { "path": "docs/talk/requirements.txt", "content": "argh==0.26.2\ndocutils==0.14\nhovercraft==2.5\nlxml==4.6.2\npathtools==0.1.2\nPygments==2.2.0\nPyYAML>=4.2b1\nsvg.path==2.2\nwatchdog==0.8.3\n" }, { "path": "docs/talk/style.css", "content": "@import url(http://fonts.googleapis.com/css?family=Vollkorn);\n\nbody {\n background-image: url(images/noise.png);\n background-color: #0094d0;\n color: #eee;\n}\n\na {\n color: #aaaaee;\n}\n\nul {\n\tfont-family: Vollkorn;\n\tfont-size: 45px;\n\tfont-style: normal;\n\tfont-variant: normal;\n\tfont-weight: 400;\n\n display: block;\n margin: auto;\n margin-bottom: 10px;\n width: fit-content;\n}\n\nol {\n\tfont-family: Vollkorn;\n\tfont-size: 30px;\n\tfont-style: normal;\n\tfont-variant: normal;\n\tfont-weight: 400;\n\n display: block;\n margin: auto;\n margin-bottom: 10px;\n width: fit-content;\n min-width: 25vw;\n}\n\nh1, h2, h3 {\n display: block;\n margin: auto;\n margin-bottom: 30px;\n width: fit-content;\n\n padding-left: 5px;\n padding-right: 10px;\n padding-top: 10px;\n\n background-color: white;\n color: #124255;\n\n\tfont-family: Vollkorn;\n\tfont-size: 60px;\n\tfont-style: normal;\n\tfont-variant: normal;\n\tfont-weight: 700;\n\n}\n\nh2 {\n\tfont-size: 40px;\n}\n\nimg {\n display: block;\n margin: auto;\n}\n\np {\n\tfont-family: Vollkorn;\n\tfont-size: 44px;\n\tfont-style: normal;\n\tfont-variant: normal;\n\tfont-weight: 400;\n\tline-height: 50px;\n\n display: block;\n margin: auto;\n margin-top: 3vh;\n margin-bottom: 3vh;\n width: fit-content;\n}\n\nblockquote {\n\tfont-family: Vollkorn;\n\tfont-size: 40px;\n\tfont-style: normal;\n\tfont-variant: normal;\n\tfont-weight: 400;\n\tline-height: 30px;\n}\n\npre {\n\tfont-family: Fira Code;\n\tfont-size: 25px;\n\tfont-style:normal;\n\tfont-weight: 400;\n\tline-height: 40px;\n\n padding: 20px;\n\n /*\n background-color: #000;\n border-color: #000;\n */\n\n border-style: solid;\n border-radius: 10px;\n\n display: block;\n margin-right: 30vw;\n min-width: 40vw;\n\n color: #0094d0;\n background-color: #fff;\n border-color: #fff;\n}\n\n/* Special roles */\n\n.slide-number {\n\t;font-family: Vollkorn;\n\tfont-style: italic;\n\tfont-size: 100px;\n\tfont-weight: 700;\n opacity: 0.3;\n}\n\n.title-logo {\n\tfont-size: 100px;\n\tfont-weight: 700;\n display: block;\n text-align: center;\n}\n\n.donald {\n font-variant: small-caps;\n\tfont-weight: 700;\n font-size: 40px;\n\n padding-left: 12px;\n padding-right: 12px;\n padding-top: 12px;\n padding-bottom: 5px;\n\n border-width: 12px;\n border-style: solid;\n border-color: white;\n}\n\n.white-bg {\n background-color: white;\n color: #124255;\n\tfont-size: 40px;\n\tfont-weight: 700;\n\n padding-left: 5px;\n padding-right: 5px;\n padding-top: 5px;\n\n display: block;\n margin: auto;\n margin-bottom: 10px;\n width: fit-content;\n}\n\n.strike {\n text-decoration: line-through;\n}\n\n.github:before {\n background-image: url(images/github.png);\n background-size: 1em 1em;\n background-repeat: no-repeat;\n display: inline-block;\n width: 1em;\n height: 1em;\n position: relative;\n top: 0.2em;\n padding-right: 0.3em;\n content: \" \";\n}\n\n.www:before {\n background-image: url(images/www.png);\n background-size: 1em 1em;\n background-repeat: no-repeat;\n display: inline-block;\n width: 1em;\n height: 1em;\n position: relative;\n top: 0.2em;\n padding-right: 0.3em;\n content: \" \";\n}\n\n.rtd:before {\n background-image: url(images/rtd.png);\n background-size: 1em 1em;\n background-repeat: no-repeat;\n display: inline-block;\n width: 1em;\n height: 1em;\n position: relative;\n top: 0.2em;\n padding-right: 0.3em;\n content: \" \";\n}\n\n.workflow {\n position: fixed;\n bottom: -15vh;\n right: 0vw;\n}\n\n.wollmilchsau {\n position: fixed;\n bottom: 5vh;\n right: -12vw;\n}\n\n.img-tux {\n position: fixed;\n bottom: 10vh;\n right: -12vw;\n}\n\n.img-gopher {\n position: fixed;\n top: 10vh;\n left: -12vw;\n}\n\n.small-list > ul {\n font-size: 40px;\n}\n\n.help {\n font-size: 12px;\n}\n\n.small {\n font-size: 32px;\n font-style: italic;\n}\n\n.underline {\n text-decoration: underline;\n font-style: italic;\n}\n" }, { "path": "docs/tutorial/config.rst", "content": ".. _configurations:\n\nConfiguration\n-------------\n\nAs mentioned earlier, we can use the built-in configuration system to configure many aspects\nof ``brig`` functionality to our liking. Every config entry of ``brig`` consists of 4 values:\n\n* Key - always a dotted, hierarchical path like ``fs.sync.ignore_moved``.\n* Value - some value that is validated depending on the key.\n* Default - The default value.\n* Documentation - A short description of what this entry can do for you.\n* Needs restart - A boolean indicating whether you have to restart the service to take effect.\n\nWhen you type ``brig cfg`` you will see all keys with the aforementioned entries:\n\n.. code-block:: bash\n\n $ brig config ls\n [...]\n fs.sync.ignore_moved: false (default)\n Default: false\n Documentation: Do not move what the remote moved\n Needs restart: no\n [...]\n\nAdditionally, we support of course the usual operations:\n\n.. code-block:: bash\n\n $ brig config get repo.password_command\n pass brig/repo/password\n $ brig config set repo.password_command \"pass brig/repo/my-password\"\n\nProfiles\n~~~~~~~~\n\n.. todo:: Implement configuration profiles.\n" }, { "path": "docs/tutorial/coreutils.rst", "content": "Adding & Viewing files\n----------------------\n\nNow let's add some files to ``brig``. We do this by using ``brig stage``. It's\ncalled ``stage`` because all files first get added to a staging area. If you\nwant, and are able to remember that easier, you can also use ``brig add``.\n\n.. code-block:: bash\n\n $ echo \"Hello World\" > /tmp/hello.world\n $ brig stage /tmp/hello.world\n $ brig cat hello.world\n Hello World\n $ brig ls\n SIZE MODTIME PATH PIN\n 986 B Mon Mar 4 23:04:07 CET 2019 /README.md ✔\n 12 B Mon Mar 4 23:04:23 CET 2019 /hello.world ✔\n\nThis adds the content of ``/tmp/hello.world`` to a new file in ``brig`` called\n``/hello.world``. The name was automatically chosen from looking at the base\nname of the added file. All files in ``brig`` have their own name, possibly\ndiffering from the content of the file they originally came from. Of course,\nyou can also add whole directories.\n\n.. note::\n\n ``brig`` always copy the data. If you happen to change the original file,\n the change will not progpagate to the file in ``brig``. You have to re-stage it\n to reflect the change.\n\nIf you want to use a different name, you can simply pass the new name as second\nargument to ``stage``:\n\n.. code-block:: bash\n\n $ brig stage /tmp/hello.world /hallo.welt\n\nYou also previously saw ``brig cat`` which can be used to get the content of\na file again. ``brig ls`` in contrast shows you a list of currently existing\nfiles, including their size, last modification time, path and pin state [#]_.\n\nOne useful feature of ``brig cat`` is that you can output directories as well.\nWhen specifying a directory as path, a ``.tar`` archive is being outputted.\nYou can use that easily to store whole directories on your disk or archive\nin order to send it to some client for example:\n\n.. code-block:: bash\n\n # Create a tar from root and unpack it to the current directory.\n $ brig cat | tar xfv -\n # Create .tar.gz out of of the /photos directory.\n $ brig cat photos | gzip -f > photos.tar.gz\n\n.. [#] Pinning and pin states are explained :ref:`pinning-section` and are not important for now.\n\nCoreutils\n---------\n\nYou probably already noticed that a lot of commands you'd type in a terminal on\na normal day have a sibling as ``brig`` command. Here is a short overview of\nthe available commands:\n\n.. code-block:: bash\n\n $ brig mkdir photos\n $ brig touch photos/me.png\n $ brig tree\n • ✔\n ├── hello.world ✔\n ├── photos/ ✔\n │ └── me.png ✔\n └── README.md ✔\n\n 2 directories, 2 files\n $ brig cp photos/me.png photos/moi.png\n $ brig mv photos/me.png photos/ich.png\n # NOTE: There is no \"-r\" switch. Directories are always deleted recursively.\n $ brig rm photos\n\nPlease refer to ``brig help `` for more information about those. They\nwork in most cases like their pendant. Also note that there is no ``brig cd``\ncurrently. All paths must be absolute.\n\nHints - Configuring encryption & compression\n--------------------------------------------\n\nOften times you might want not encrypt all files. A typical use case would be\nto have a ``/public`` folder where you put in files to share with your friends.\nProbably there are some freely available files in there, you got from some\ncorners of the internet (for example your excellent meme collection). Those\nfiles don't need encryption and probably not even compression. If you want to\nexclude the ``/public`` folder from both you can give ``brig`` a hint:\n\n.. code-block:: bash\n\n # let's assume /public exists already:\n $ brig hints set /public --compression none --encryption none\n $ brig hints\n PATH ENCRYPTION COMPRESSION\n / aes256gcm guess\n /public none none\n\n\nAs you might notice, there is already one hint set by default for the root directory.\nIf you want to change the global defaults, you can simply modify this one. Below you\nsee the hint you just created. This however does not change any existing files. It just\ntells ``brig`` »next time you modify those files, please use those algorithms«. If you\nwant to make sure the files are changed to use the algorithm you set, then you can\nuse the ``stage --recode`` command:\n\n.. code-block:: bash\n\n $ brig stage --recode /public\n\n\nIf you do this, you can observe a small change when looking at the ``IsRaw``\nattribute of the file's info:\n\n.. code-block:: bash\n\n # This was 'true' before the recode.\n $ brig info --format '{{ .IsRaw }}' /public/cat-meme.png\n false\n\nThe ``IsRaw`` attribute tells you if you could download this file by its hash\nfrom an IPFS gateway. If its true, ``brig`` does not touch it at all. This is\nan useful attribute you want to share a file with your non-tech friends who\nprefer to click on a regular HTTP URL: You can just point them a [IPFS\ngateway](https://docs.ipfs.io/concepts/ipfs-gateway).\n\nAvailable encryption algorithms\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n.. note::\n\n The ``THROUGHPUT`` numbers shows the relative, average performance compared\n to ``none``. Your mileage may vary a lot. Those number should serve as\n rough guideline and were obtained by the built-in ``briog debug iobench``\n utility using the ``fuse-{read,write}-mem`` benchmark. If you want the\n details you can run the benchmarks yourself. As you can see from the numbers,\n the additional encoding by brig does not make things substantially slower.\n\n If you wonder how some benchmark are faster than ``none``: Compression\n compacts the stream heavily (if the data is well compressible). Therefore\n less bytes need to be transferred and encrpyted or decrypted. Quite surprisingly,\n in some cases compression can make things faster.\n\n Also note that this was measured without caching. If no data is modified\n your operating system will likely cache data for you and speed up things.\n\n+----------------+----------------------------------------------------------------------------+--------------------+-----------------+\n| NAME | DESCRIPTION | READ THROUGHPUT | WRITE THROGHPUT |\n+================+============================================================================+====================+=================+\n| ``aes256-gcm`` | The default. AES with 256 bit key in GCM cipher mode. Fast on modern CPUs. | 80-85% | 85-95% |\n+----------------+----------------------------------------------------------------------------+--------------------+-----------------+\n| ``chacha20`` | Streaming cipher with Poly1305 MAC. Good for old CPUs without AES-NI. | 70-85% | 80-90% |\n+----------------+----------------------------------------------------------------------------+--------------------+-----------------+\n| ``none`` | Disables encryption. Fast, but only good for public files. | 100% | 100% |\n+----------------+----------------------------------------------------------------------------+--------------------+-----------------+\n\n\nAvailable compression algorithms\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n+----------------+----------------------------------------------------------------------------+-------------------+---------------------+\n| NAME | DESCRIPTION | READ THROUGHPUT | WRITE THROUGHPUT |\n+================+============================================================================+===================+=====================+\n| ``snappy`` | High throughput, relative low compression ratio. | 80-105% | 95-130% |\n+----------------+----------------------------------------------------------------------------+-------------------+---------------------+\n| ``lz4`` | Middle throughput, slightly higher compression ratio than snappy. | 77-93% | 85-105% |\n+----------------+----------------------------------------------------------------------------+-------------------+---------------------+\n| ``zstd`` | Low throughput, highest compression ratio. | 55-95% | 35-100% |\n+----------------+----------------------------------------------------------------------------+-------------------+---------------------+\n| ``guess`` | Chooses suitable algorithm based on file ending, size and mime type. | ``-`` | ``-`` |\n+----------------+----------------------------------------------------------------------------+-------------------+---------------------+\n| ``none`` | Disables compression. | 100% | 100% |\n+----------------+----------------------------------------------------------------------------+-------------------+---------------------+\n" }, { "path": "docs/tutorial/gateway.rst", "content": "Using the gateway / UI\n----------------------\n\nGateway Screenshots\n~~~~~~~~~~~~~~~~~~~\n\nThe gateway UI consists of several tabs, which are briefly shown below to give you a short impression of it.\n\nLogin screen\n^^^^^^^^^^^^\n\nAllows you to login. You can also come back here to change the user.\nIt is also possible to login anonymously, as you will see below.\n\n.. image:: ../_static/gateway-login.png\n :alt: Gateway login screen\n :width: 66%\n\nFile Browser\n^^^^^^^^^^^^\n\nThe main view. Lists the directory tree and file attributes.\nAllows for modification, uploading and everything what you'd expect.\n\n.. image:: ../_static/gateway-files.png\n :alt: Gateway files view\n :width: 66%\n\nChangelog View\n^^^^^^^^^^^^^^\n\nA list of commits. You are able to jump back to a specific commit.\n\n.. image:: ../_static/gateway-changelog.png\n :alt: Gateway changelog view\n :width: 66%\n\nTrashbin\n^^^^^^^^\n\nA list of deleted files. If you deleted something you will be able to get it back here.\n\n.. image:: ../_static/gateway-trashbin.png\n :alt: Gateway trashbin view\n :width: 66%\n\nRemote List\n^^^^^^^^^^^\n\nIf your user is privileged enough, you can see and edit the list of remotes and\nadjust settings in it.\n\n.. image:: ../_static/gateway-remotes.png\n :alt: Gateway remotes view\n :width: 66%\n\nRemote Add Dialog\n^^^^^^^^^^^^^^^^^\n\nA sample dialog. The UI uses many of them.\n\n.. image:: ../_static/gateway-add-remote.png\n :alt: Gateway add remote view\n :width: 66%\n\n---------\n\nIntroduction\n~~~~~~~~~~~~\n\nMany users will not run ``brig`` themselves, so you won't be able to ``brig\nsync`` with them. Chances are that you still want to send or present them your\nfiles without too much hassle. ``brig`` features a *Gateway* to HTTP(S), which\ncomes particularly handy if you happen to run a public server and/or want to\nprovide a GUI to your users. It also includes an easy to use UI that is enabled\nby default.\n\nBefore you do anything, you need to a »user« to your gateway. This user is different\nthan remotes and describes what credentials can be used to access the gateway.\nYou can add add a new user like this:\n\n.. code-block:: bash\n\n $ brig gateway user add admin my-password\n # or shorter:\n # brig gw u a admin my-password\n $ brig gateway user list\n NAME FOLDERS\n admin /\n\nThe gateway is disabled by default. If you want to start it, use this command:\n\n.. code-block:: bash\n\n $ brig gateway start\n\nWithout further configuration, this will create a HTTP (**not HTTPS!**) server\non port ``6001``, which can be used already. If you access it under\n``http://localhost:6001`` you will see a login mask where you can log yourself\nin with the credentials you entered earlier.\n\nIf you'd like to use another port than ``6001``, you can do so by setting the\nrespective config key:\n\n.. code-block:: bash\n\n $ brig cfg set gateway.port 7777\n\n.. note::\n\n You can always check the status of the gateway:\n\n .. code-block:: bash\n\n $ brig gateway status\n\n This will also print helpful diagnostics if something might be wrong.\n\nThe gateway can be stopped anytime with the following command:\n\n.. code-block:: bash\n\n $ brig gateway stop\n\nThere is also a small helper that will print you a nice hyperlink to a certain\nfile called ``brig gateway url``:\n\n.. code-block:: bash\n\n $ brig gateway url README.md\n http://localhost:6001/get/README.md\n\nFolder management\n~~~~~~~~~~~~~~~~~\n\nYou probably do not want to offer your files to everyone that have a link.\nTherefore you can restrict access to a few folders (``/public`` for example)\nfor individual users. By default a user is allowed to see everything. If you want\na user that can only access the ``/public`` folder simply add him as follows:\n\n.. code-block:: bash\n\n $ brig gw user add my-new-user /public\n\nNow only the files in ``/public`` (and including ``/public`` itself) are\naccessible from the gateway.\n\nUser right management\n~~~~~~~~~~~~~~~~~~~~~\n\nWe already discussed the adding of a user above. There is a little more to that though.\nYou can add users with different rights. In total there are 5 different rights currently:\n\n* **fs.view**: View and list all files.\n* **fs.edit**: Edit and create new files.\n* **fs.download**: Download file content.\n* **remotes.view**: View the remotes tab.\n* **remotes.edit**: Edit the remotes tab.\n\nWhen you add users you can give a new user a comma separated list of rights via the ``-r`` switch:\n\n.. code-block:: bash\n\n $ brig gw user add my-new-user -r 'remotes.view,remotes.edit'\n\nFor your convenience there are a bunch of presets which will do the work for you in 99% of the cases:\n\n* ``--role-admin, -a``: Add this user as admin (short for »-r 'fs.view,fs.edit,fs.download,remotes.view,remotes.edit'«)\n* ``--role-editor, -b``: Add this user as collaborator (short for »-r 'fs.view,fs.edit,fs.download,remotes.view'«)\n* ``--role-collaborator, -c``: Add this user as collaborator (short for »-r 'fs.view,fs.edit,fs.download'«)\n* ``--role-viewer, -d``: Add this user as viewer (short for »-r 'fs.view,fs.download'«)\n* ``--role-link-only, -e``: Add this user as linker (short for »-r 'fs.download'«)\n\nRunning the gateway with HTTPS\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nBy default, we run with ``http`` only. If you want to expose the gateway under\na domain to the internet you should secure it with ``https``. Since brig's\ngateway might is likely not the only service you want to expose we recommend a\nwebserver like `Caddy`_ (which is great software in general!) that\nautomatically fetches certificates and reverse-proxies traffic to the gateway.\nA minimal ``Caddyfile`` can look like this:\n\n.. code-block::\n\n # Assumption:\n # brig gateway runs on the same server as Caddy\n # and is reachable under localhost:6001.\n your.domain.org {\n route /gateway/* {\n uri strip_prefix /gateway\n reverse_proxy http://localhost:6001\n }\n }\n\nWith this setup, your gateway would be reachable under\n``https://your.domain.org/gateway``. You can of course choose a different\nroute or even a sub-domain. Maybe you also want to setup compression or require\na client certificate. Refer to the Caddy documentation for more information.\n\n.. _Caddy: https://caddyserver.com/docs/getting-started\n\nAllowing anonymous access\n~~~~~~~~~~~~~~~~~~~~~~~~~\n\nIf you want to run a public gateway (for example for a group of friends), then\nyou might want to enable anonymous access. In this mode you will be logged in\nright away to the gateway without facing the login screen. You still have the\noption to go to the login screen and become another user.\n\nYou can enable the anonymous mode like this:\n\n.. code-block:: bash\n\n $ brig cfg set gateway.auth.anon_allowed true\n\nAdditionally you have to create an ``anon`` user. This allows you to define\nwhat rights the anonymous users have and what folders they may access:\n\n.. code-block:: bash\n\n # Give the anonymous users only access to /public and don't let them modify anything:\n $ brig gw u add anon anon --role-viewer /public\n\n\nIf you want to change the name of the ``anon`` user to something else (for whatever reason) you can do so by\nsetting the ``auth.anon_user`` variable. You also have to re-add the user above with the new name.\n\n.. code-block:: bash\n\n $ brig cfg set gateway.auth.anon_user some_other_anon_name_that_is_not_used\n" }, { "path": "docs/tutorial/init.rst", "content": "Creating a repository\n---------------------\n\nYou need a central place where ``brig`` stores its metadata. This place is\ncalled a »repository« or short »repo«. This is not the place, where your files\nare stored. Those are copied (if you did setup IPFS in a normal way) to\n``~/.ipfs``. Keep in mind that ``brig`` will copy files and thus will never\nmodify the original files on your hard drive.\n\nBy creating a new repository you also generate your identity, under which your\nbuddies can later **find** and **authenticate** you. But enough of the mere\ntheory, let's get started:\n\n.. code-block:: bash\n\n # Create a place where we store our metadata.\n # The repository is created by default in the current working directory.\n # (This can be changed via `brig --repo`)\n\n $ mkdir repo && cd repo\n $ brig init ali@woods.org/desktop -w 'echo my-password'\n\n _____ / /\\ ___ / /\\\n / /::\\ / /::\\ / /\\ / /:/_\n / /:/\\:\\ / /:/\\:\\ / /:/ / /:/ /\\\n / /:/~/::\\ / /:/~/:/ /__/::\\ / /:/_/::\\\n /__/:/ /:/\\:| /__/:/ /:/___ \\__\\/\\:\\__ /__/:/__\\/\\:\\\n \\ \\:\\/:/~/:/ \\ \\:\\/:::::/ \\ \\:\\/\\ \\ \\:\\ /~~/:/\n \\ \\::/ /:/ \\ \\::/~~~~ \\__\\::/ \\ \\:\\ /:/\n \\ \\:\\/:/ \\ \\:\\ /__/:/ \\ \\:\\/:/\n \\ \\::/ \\ \\:\\ \\__\\/ \\ \\::/\n \\__\\/ \\__\\/ \\__\\/\n\n\n A new file README.md was automatically added.\n Use 'brig cat README.md' to view it & get started.\n\n $ ls\n config.yml gateway immutable.yml keyring\n metadata README.md remotes.yml\n\nThe name you specified after the ``init`` is the name that will be shown\nto other users and by which you are searchable in the network.\nSee :ref:`about_names` for more details on the subject.\n\nOnce the ``init`` ran successfully there will be a daemon process running in\nthe background. Every other ``brig`` commands will communicate with it via\na local network socket. If the daemon does not run yet, it will be started for\nyou in the background without you noticing.\n\n.. note::\n\n If no IPFS daemon is running, ``brig`` will start one for you. If you don't\n have ``ipfs`` installed, it will even install and set it up for you. By\n default, ``brig init`` will also set some default options that help ``brig``\n to run a bit smoother. If you do not want those, please add\n ``--no-ipfs-optimization`` to the ``init`` command above.\n\n.. _about_names:\n\nChoosing and finding names\n~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nYou might wonder what the name you pass to ``init`` is actually for. As\npreviously noted, there is no real restriction for choosing a name, so all of\nthe following are indeed valid names:\n\n- ``ali``\n- ``ali@woods.org``\n- ``ali@woods.org/desktop``\n- ``ali/desktop``\n\nIt's however recommended to choose a name that is formatted like\na XMPP/Jabber-ID. Those IDs can look like plain emails, but can optionally have\na »resource« part as suffix (separated by a »/« like ``desktop``). Choosing\nsuch a name has two advantages:\n\n- Other peers can find you by only specifying parts of your name.\n Imagine all of the *Smith* family members use ``brig``, then they'd possibly those names:\n\n * ``dad@smith.org/desktop``\n * ``mom@smith.org/tablet``\n * ``son@smith.org/laptop``\n\n When ``dad`` now sets up ``brig`` on his server, he can use ``brig net locate\n -m domain 'smith.org'`` to get all fingerprints of all family members. Note\n however that ``brig net locate`` **is not secure**. Its purpose is solely\n discovery, but is not able to verify that the fingerprints really correspond\n to the persons they claim to be. This due to the distributed nature of\n ``brig`` where there is no central or federated authority that coordinate\n user name registrations. So it is perfectly possible that one name can be\n taken by several repositories - only the fingerprint is unique.\n\n- Later development of ``brig`` might interpret the user name and domain as\n email and might use your email account for verification purposes.\n\nHaving a resource part is optional, but can help if you have several instances\nof ``brig`` on your machines. i.e. one user name could be\n``dad@smith.org/desktop`` and the other ``dad@smith.org/server``.\n\n\nRunning the daemon and viewing logs\n-----------------------------------\n\nThe following sections are not a required read. They are useful to keep in\nmind, but in the ideal case you're don't even need to think about the daemon.\n\nAs discussed before, the daemon is being started on demand in the background.\nSubsequent commands will then use the daemon. For debugging purposes it can be useful\nto run in the daemon in the foreground. You can do this with the ``brig daemon`` commands:\n\n.. code-block:: bash\n\n # Make sure no prior daemon is running:\n $ brig daemon quit\n # Start the daemon in the foreground and log to stdout:\n $ brig daemon launch -s\n\nIf you want to quit the instance, either just hit CTRL-C or type ``brig daemon\nquit`` into another terminal window.\n\nLogging\n~~~~~~~\n\nUnless you pass the ``-s`` (``--log-to-stdout`` flag) as above, all logs are\nbeing piped to the system log. You can follow the log like this:\n\n.. code-block:: bash\n\n # Follow the actual daemon log:\n $ journalctl -ft brig\n\nThis assumes you're using a ``systemd``-based distribution. If not, refer to\nthe documentation of your syslog daemon.\n\nUsing several repositories in parallel\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nIt can be useful to run more than one instance of the ``brig`` daemon in\nparallel. Either for testing purposes or as actual production configuration. In\norder for the ``brig`` client to know what daemon to talk to, you have to be\nspecific about the repository (``--repo``) path. Here is an example:\n\n.. code-block:: bash\n\n # Be explicit\n $ brig --repo /tmp/ali init ali -x --ipfs-path ~/.ipfs\n $ brig --repo /tmp/bob init bob -x --ipfs-path ~/.ipfs2\n\n # Since you specified --repo we know what daemon to talk to.\n # You can also set BRIG_PATH for the same effect:\n $ BRIG_PATH=/tmp/ali brig ls\n \n\n # Add some alias to your .bashrc to save you some typing:\n $ alias brig-ali=\"brig --repo /tmp/ali\"\n $ alias brig-bob=\"brig --repo /tmp/bob\"\n\n # Now you can use them normally,\n # e.g. by adding them as remotes each:\n $ brig-ali remote add bob $(brig-bob whoami -f)\n $ brig-bob remote add ali $(brig-ali whoami -f)\n\n\n.. note::\n\n It is possible to have several repositories per IPFS instances. Since things\n might get confusing though when it comes to pinning, it is recommended to\n have several IPFS daemons running in this case. This is done via the\n ``--ipfs-port`` flag in the example above.\n\nLocking the repository.\n-----------------------\n\nThe repository on disk is not encrypted. If you plan on moving the repository\nto somewhere else, e.g. by copying it onto an USB stick and physically moving\nit somewhere else you should always consider to first create an encrypted\narchive out of it and unpack it on the target machine. ``brig`` has a built-in\nhelper for this. Please refer to ``brig pack-repo --help`` and ``brig\nunpack-repo --help``.\n" }, { "path": "docs/tutorial/intro.rst", "content": ".. _getting_started:\n\nGetting started\n================\n\nThis guide will walk you through the steps of synchronizing your first files\nover ``brig``. You will learn about the concepts behind it along the way. Most\nof the steps here will include working in a terminal, since this is the primary\nway to interact with ``brig``. Once setup you have to choice to use a browser\napplication though.\n\nPrecursor: The help system\n--------------------------\n\nBefore we dive in, we go over a few things that will make your life easier\nalong the way. ``brig`` has some built-in helpers to serve as support for your\nmemory. If you're not interested in that you can skip right to the next section.\nBut please check those help texts before asking questions.\n\nBuilt-in reference documentation\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nEvery command offers detailed built-in help, which you can view using the\n``brig help`` command. This often usage examples too:\n\n.. code-block:: bash\n\n $ brig help stage\n NAME:\n brig stage - Add a local file to the storage\n\n USAGE:\n brig stage [command options] ( []|--stdin )\n\n CATEGORY:\n WORKING TREE COMMANDS\n\n DESCRIPTION:\n Read a local file (given by »local-path«) and try to read\n it. This is the conceptual equivalent of »git add«. [...]\n\n EXAMPLES:\n\n $ brig stage file.png # gets added as /file.png\n $ brig stage file.png /photos/me.png # gets added as /photos/me.png\n $ cat file.png | brig stage --stdin /file.png # gets added as /file.png\n\n OPTIONS:\n --stdin, -i Read data from stdin\n\nShell autocompletion\n~~~~~~~~~~~~~~~~~~~~\n\n.. warning::\n\n The shell autocompletion is still under development. It might still yield\n weird results and the usability needs to be improved definitely. Any help\n welcome!\n\nIf you don't like to remember the exact name of each command, you can use\nthe provided autocompletion. For this to work you have to insert this\nat the end of your ``.bashrc``:\n\n.. code-block:: bash\n\n source $GOPATH/src/github.com/sahib/brig/autocomplete/bash_autocomplete\n\nOr if you happen to use ``zsh``, append this to your ``.zshrc``:\n\n.. code-block:: bash\n\n source $GOPATH/src/github.com/sahib/brig/autocomplete/zsh_autocomplete\n\nAfter starting a new shell you should be able to autocomplete most commands.\nTry this for example by typing ``brig remote ``. Other shells are not\nsupported right now sadly.\n\nOpen the online documentation\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nBy typing ``brig docs`` you'll get a tab opened in your default browser with this\ndomain loaded. Please stop typing ``brig docs`` into Google and save some energy.\n\nReporting bugs\n~~~~~~~~~~~~~~~\n\nIf you need to report a bug (thank you!) you can use a built-in utility to do\nthat. It will gather all relevant information, create a report and open a tab\nwith the *GitHub* issue tracker in a browser for you. Only thing left for you\nis to fill out some questions in the report and include anything you think is\nrelevant.\n\n.. code-block:: bash\n\n $ brig bug\n\nTo actually create the issue you sadly need an *GitHub* `account\n`_. If you don't have internet or do not want to sign\nup, you can still generate a bug report template via ``brig bug -s``.\n" }, { "path": "docs/tutorial/mounts.rst", "content": "Mounting repositories\n---------------------\n\nUsing commands like ``brig cp`` might not feel very seamless, especially when\nbeing used to tools like file browsers. And indeed, those commands are only\nsupposed to serve as a low-level way of interacting with ``brig`` and as way\nfor scripting own, more elaborate workflows.\n\nFor your daily workflow it is far easier to mount all files known to ``brig``\nto a directory of your choice and use it with the tools you are used to. To\naccomplish that ``brig`` supports a FUSE filesystem that can be controlled via\nthe ``mount`` and ``fstab`` commands. Let's look at ``brig mount``:\n\n.. code-block:: bash\n\n $ mkdir ~/data\n $ brig mount ~/data\n $ cd ~/data\n $ cat hello-world\n Hello World\n $ echo 'Salut le monde!' > salut-monde.txt\n # There is no difference between brig's \"virtual view\"\n # and the contents of the mount:\n $ brig cat salut-monde.txt\n Salut le monde!\n\nYou can use this directory like a normal one, but check for the CAVEATS below.\nYou can have any number of mounts. This proves especially useful when only\nmounting a subdirectory (let's say we have a directory called ``/Public``) with\nthe ``--root`` option of ``brig mount`` and mounting all other files as read\nonly (``--readonly``).\n\n.. code-block:: bash\n\n $ brig mount ~/data --readonly\n $ brig mkdir /writable\n $ brig touch /writable/please-edit-me\n $ mkdir ~/rw-data\n $ brig mount ~/rw-data --root /writable\n $ echo 'writable?' > ~/data/test\n read-only file system: ~/data/test\n $ echo 'writable!' > ~/rw-data/test\n $ cat ~/rw-data/test\n writable!\n\nAn existing mount can be removed again with ``brig unmount ``:\n\n.. code-block:: bash\n\n $ brig unmount ~/data\n $ brig unmount ~/rw-data\n $ brig rm writable\n\n\nRemote access\n~~~~~~~~~~~~~\n\nWorking with remote data does often not work extremely well with the file\nabstraction that does not play well with timeouts. This often causes\napplications to hang for indefinite times, since they are not most of the time\nnot build for data that might not be delivered immediately. For this very\ncommon case we have the ``--offline`` flag. It will error out immediately on\nfiles that are not in our local cache:\n\n.. code-block:: bash\n\n $ brig mount /tmp/mount --offline\n # Or with fstab:\n $ brig fstab add some-mount /tmp/mount --offline\n\nIf you have a remote file you want to read, you can do this to make it cached locally:\n\n.. code-block:: bash\n\n $ brig cat /remote-file > /dev/null\n\nAfter ``brig cat`` run, you should be able to view the file normally in the mount.\n\n.. _permanent-mounts:\n\nMaking mounts permanent\n~~~~~~~~~~~~~~~~~~~~~~~\n\nAll mounts that are created via ``brig mount`` will be gone after a daemon restart.\nIf you a typical set of mounts, you can persist them with the ``brig fstab`` facility:\n\n.. code-block:: bash\n\n $ brig fstab add tmp_rw_mount /tmp/rw-mount\n $ brig fstab add tmp_ro_mount /tmp/ro-mount -r\n $ brig fstab\n NAME PATH READ_ONLY ROOT ACTIVE\n tmp_ro_mount /tmp/ro-mount yes /\n tmp_rw_mount /tmp/rw-mount no /\n $ brig fstab apply\n $ brig fstab\n NAME PATH READ_ONLY ROOT ACTIVE\n tmp_ro_mount /tmp/ro-mount yes / ✔\n tmp_rw_mount /tmp/rw-mount no / ✔\n $ brig fstab apply -u\n NAME PATH READ_ONLY ROOT ACTIVE\n tmp_ro_mount /tmp/ro-mount yes /\n tmp_rw_mount /tmp/rw-mount no /\n\nEt Voilà, all mounts will be created and mounted once you enter ``brig fstab\napply`` or restart the daemon. The opposite can be achieved by executing ``brig\nfstab apply --unmount``.\n\n*CAVEATS:* The FUSE filesystem is not yet perfect and somewhat experimental. Keep those points in mind:\n\n- **Performance:** Writing to FUSE is currently somewhat *memory and CPU\n intensive*. Generally, reading should be fast enough for most basic use\n cases, but also is not enough for high performance needs. If you need to edit\n a file many times, it is recommended to copy the file somewhere to your local\n storage (e.g. ``brig cat the_file > /tmp/the_file``), edit it there and save\n it back for syncing purpose. Future releases will work on optimizing the\n performance.\n- **Timeouts:** Although it tries not to look like one, we're operating on\n a networking filesystem. Every file you access might come from a different\n computer. If no other machine can serve this file we might block for a long\n time, causing application hangs and general slowness. This is a problem that\n still needs a proper solution and leaves much to be desired in the current\n implementation.\n" }, { "path": "docs/tutorial/pinning.rst", "content": ".. _pinning-section:\n\nPinning\n-------\n\nHow can we control what files are stored locally and which should be retrieved\nfrom the network? You can do this by **pinning** each file or directory you\nwant to keep locally. Normally, files that are not pinned may be cleaned up\nfrom time to time, that means they are evaded from the local cache and need to\nbe fetched again from the network when being accessed again. Since you still\nhave the metadata for this file, you won't notice the difference beside some\npossible network lag. When you pin a file however, it will not be garbage\ncollected and stays in your local cache until unpinned.\n\n``brig`` knows of two types of pins: **Explicit** and **implicit**.\n\n- **Implicit pins:** This kind of pin is created automatically by ``brig`` and\n cannot be created by the user. In the command line output it is always shows as\n blue pin. Implicit pins are created by ``brig`` whenever you create a new\n file, or update the contents of a file. Implicit pins are managed by ``brig`` and\n as you will see later, it might decide to save you some space by unpinning old versions.\n- **Explicit pins:** This kind of pin is created by the user explicitly (hence\n the name) and is never done by ``brig`` automatically. It has the same effect\n as an implicit pin, but cannot be removed again by ``brig``, unless\n explicitly unpinned by the user. This is a good way of telling ``brig`` to\n never unpin this specific version. Use this with care, since it is easy to forget about\n explicit pins.\n\nWhen syncing with somebody, all files retrieved by them are by default **not\npinned**. If you want to keep them for longer, make sure to pin them\nexplicitly.\n\nGarbage collection\n~~~~~~~~~~~~~~~~~~\n\nStrongly related to pinning is garbage collection. Whenever you need to clean up some\nspace, you can just type ``brig gc`` to remove all unpinned files from the cache.\n\nBy default, the garbage collector is also run once every hour. You can change this interval\nby setting ``brig config set repo.autogc.interval`` to ``30m`` for example. You can also disable\nthis automatic garbage collection by issuing ``brig config set repo.autogc.enabled false``.\n\nRepinning\n~~~~~~~~~\n\nRepinning allows you to control how many versions of each file you want to\nstore and/or how much space you want to store at most. The repinning feature is\ncontrolled by the following configuration variables:\n\n- **fs.repin.quota**: Maximum amount of data to store in a repository.\n- **fs.repin.min_depth**: Keep this many versions definitely pinned. Trumps quota.\n- **fs.repin.max_depth**: Unpin versions beyond this depth definitely. Trumps quota.\n- **fs.repin.enabled**: Wether we should allow the repinning to run at all.\n- **fs.repin.interval**: How much time to wait between calling repinning automatically.\n\nNormally repinning will run for you every 15 minutes. You can also trigger it manually:\n\n.. code-block:: bash\n\n $ brig pin repin\n\nBy default, ``brig`` will keep 1 version definitely (**fs.repin.min_depth**)\nand delete all versions starting with the 10th (**fs.repin.max_depth**). The\ndefault quota (**fs.repin.quota**) is 5GB. If repin detects files that need to\nbe unpinned, then it will first unpin all files that are beyond the max depth\nsetting. If this is not sufficient to stay under the quota, it will delete old\nversions, layer by layer starting with the biggest version first.\n" }, { "path": "docs/tutorial/remotes.rst", "content": "Remotes\n-------\n\nUntil now, all our operations were tied only to our local computer. But\n``brig`` is a synchronization tool and that would be hardly very useful without\nsupporting other peers. We call other peers »remotes« similar to the term used\nin the ``git`` world.\n\nA remote consists of three things:\n\n- **A human readable name:** This name can be choose by the user and can take\n pretty much any form, but we recommend to sticking for a form that resembles\n an extended email [#]_ like »ali@woods.org/desktop«. This name is **not**\n guaranteed to be unique! In theory everyone could take it and it is therefore\n only used for display purposes. There is no central place where users are registered.\n- **A unique fingerprint:** This serves both as address for a certain\n repository and as certificate of identity. It is long and hard to remember,\n which is the reason why ``brig`` offers to loosely link a human readable to\n it.\n- **A bunch of settings and state:** ``brig`` knows about every remote if it is\n online and/or authenticated. Additionally you can set a few remote-specific\n configuration settings like automatic updating.\n\n.. [#] To be more exact, it resembles an `XMPP or Jabber-ID `_.\n\nIf we want to find out what *our own* name and fingerprint is, we can use the\n``brig whoami`` command to ask a very existential questions:\n\n.. code-block:: bash\n\n # NOTE: The hash will look different for you:\n $ brig whoami\n ali@woods.org/desktop QmTTJbkfG267gidFKfDTV4j1c843z4tkUG93Hw8r6kZ17a:W1nayTG5UMcVxy9mFFNjuZDUb7uVTnmwFYiJ4Ajr1TP3bg\n\n.. note::\n\n The fingerprint consists of two hashes divided by a colon (:). The first\n part is the identity of your ``IPFS`` node, the second part is the\n fingerprint of a keypair that was generated by ``brig`` during init and\n will be used to authenticate other peers.\n\nWhen we want to synchronize with another repository, we need to exchange fingerprints\nand each other as remote. There are three typical scenarios here:\n\n1. Both repositories are controlled by you. In this case you can simple execute\n ``brig whoami`` on both repositories and add them with ``brig remote add``\n as described in the following.\n2. You want to sync with somebody you know. In this case you should both\n execute ``brig whoami`` and send its output over a trusted side channel.\n Personally, I use a `secure messenger like Signal `_,\n but you can also use any channel you like, including encrypted mail or\n meeting up with the person in question.\n3. You don't know each other: Get to know each other and the proceed like in\n the second point. There is no way to know if somebody is the person he is\n pretending to be, so validate that over a separate channel - that's sadly\n something where ``brig`` can't help you yet.\n\n If you need to get a hint of what users use a certain domain,\n you can use ``brig net locate`` to get a list of those:\n\n .. code-block:: bash\n\n # This command might take some time to yield results:\n $ brig net locate -m domain woods.org\n NAME TYPE FINGERPRINT\n ali@woods.org domain QmTTJbk[...]:W1UDvKzjRPb4rbbk[...]\n\n Please note again: Do not blindly add the fingerprint you see here. Always\n make sure the person you're syncing with is the one you think they are.\n\n .. todo::\n\n This seems currently broken as it does not yield any results.\n\nOnce you have exchanged the fingerprints, you add each other as **remotes**.\nLet's call the other side *bob*: [#]_\n\n.. code-block:: bash\n\n\t$ brig remote add bob \\\n\t\tQmUDSXt27LbCCG7NfNXfnwUkqwCig8RzV1wzB9ekdXaag7:\n\t\tW1e3rNGGCuuQnzyoiBKLdoN41yQ4NfNy9nRD3MwXk6h8Vy\n\n\n.. [#] The name you choose as remote can be anything you like and does not need\n to match the name the other person chose for themselves. It's not a bad\n idea though.\n\n*Bob* has do the same on his side. Otherwise the connection won't be\nestablished, because the other side won't be authenticated. By adding somebody\nas remote we **authenticate** them:\n\n.. code-block:: bash\n\n\t$ brig remote add ali \\\n QmTTJbkfG267gidFKfDTV4j1c843z4tkUG93Hw8r6kZ17a:\n W1nayTG5UMcVxy9mFFNjuZDUb7uVTnmwFYiJ4Ajr1TP3bg\n\nThanks to the fingerprint, ``brig`` now knows how to reach the other repository\nover the network. This is done in the background via IPFS and might take a few\nmoments until a valid route to the host was found.\n\nThe remote list can tell us if a remote is online:\n\n.. code-block:: bash\n\n $ brig remote list\n NAME FINGERPRINT ROUNDTRIP ONLINE AUTHENTICATED LASTSEEN AUTO-UPDATE\n bob QmUDSXt27 0s ✔ ✔ Apr 16 17:31:01 no\n $ brig remote ping bob\n ping to bob: ✔ (0.00250s)\n\nNice. Now we know that bob is online (✔) and also that he authenticated us (✔).\nOtherwise ``brig remote ping bob`` would have failed.\n\n.. note:: About open ports:\n\n While ``ipfs`` tries to do it's best to avoid having the user to open ports\n in his firewall/router. This mechanism might not be perfect though and maybe\n never is. If any of the following network operations might not work it might\n be necessary to open the port 4001 and/or enable UPnP. For security reasons\n we recommend to only open the required ports explicitly and not to use UPnP\n unless necessary.\n\nSyncing\n-------\n\nNow that we added a remote, a whole new set of features are available to us.\nBefore we move on to do our first synchronization, let's do a quick recap of\nwhat we have done so far:\n\n- Create a repository (``brig init ``) - This needs to be done only once.\n- Create optional mount points (``brig fstab add ``) - This needs to be done only once.\n- Find & add remotes (``brig remote add``) - This needs to be done once for each peer.\n- Add some files (``brig stage ``) - Do as often as you like.\n\nAs you can see, there is a bit of initial setup work, but the actual syncing is\npretty effortless now. Before we attempt to sync with anybody, it's always\na good idea to see what changes they have. We can check this with ``brig diff\n``:\n\n.. code-block:: bash\n\n # The \"--missing\" switch also tells us what files the remote does not possess:\n $ brig diff bob --missing\n •\n ├── _ hello.world\n ├── + videos/\n └── README.md ⇄ README.md\n\nThis output resembles the one we saw from ``brig tree`` earlier. Each node in\nthis tree tells us about something that would happen when we merge. The prefix\nof each file and the color in the terminal indicate what would happen with this\nfile. Refer to the table below to see what prefix relates to what action:\n\n====== ====================================================================\nSymbol Description\n====== ====================================================================\n``+`` This file is only present on the remote side.\n``-`` This file was removed on the remote side.\n``→`` This file was moved to a new location.\n``*`` This file was ignored because we chose to, due to our settings.\n``⇄`` Both sides have changes, but they are compatible and can be merged.\n``⚡`` Both sides have changes, but they are incompatible and result in conflict files.\n``_`` This file is missing on the remote side (needs to be enabled with ``--missing``)\n====== ====================================================================\n\n.. note::\n\n Remember that ``brig`` does not do any actual diffs between files, i.e. it\n will not show you what line changed. It does not care a lot about the\n content. It only records how the file metadata changes and what content\n hash the file has at a certain point.\n\nIf you prefer a more traditional view, similar to ``git``, you can use\n``--list`` on ``brig diff``.\n\nSo in the above output we can tell that *Bob* added the directory\n``/videos``, but does not possess the ``/hello.world`` file. He also\napparently modified ``README.md``, but since we did not, it's safe for us to\ntake over his changes. If we sync now we will get this directory from him:\n\n.. code-block:: bash\n\n $ brig sync bob\n $ brig ls\n SIZE MODTIME OWNER PATH PIN\n 443 B Dec 27 14:44:44 ali /README.md 🖈\n 443 B Dec 27 14:44:44 bob /README.md.conflict.0\n 12 B Dec 27 15:14:16 ali /hello.world 🖈\n 32 GB Dec 27 15:14:16 bob /videos\n\nYou might notice that the ``sync`` step took only around one second, even\nthough ``/videos`` is 32 GB in size. This is because ``sync`` **does not\ntransfer actual data**. It only transferred the metadata, while the actual data\nwill only be loaded when required. This might sound a little inconvenient at\nfirst. When I want to watch the video, I'd prefer to have it cached locally\nbefore viewing it to avoid stuttering playback. If you plan to use the files\nimmediately, you should be using pinning (see :ref:`pinning-section`)\n\nData retrieval\n~~~~~~~~~~~~~~\n\nIf the data is not on your local machine, where is it then? Thanks to IPFS\nit can be transferred from any other peer that caches this particular content.\nContent is usually cached when the peer either really stores this file or if\nthis peer recently used this content. In the latter case it will still be\navailable in its cache. This property is particularly useful when having a\nsmall device for viewing data (e.g. a smartphone, granted ``brig`` would run\nthere) and a big machine that acts as storage server (e.g. a desktop).\n\nHow are the files secure then if they essentially could be everywhere? Every\nfile is encrypted by ``brig`` before giving it to IPFS. The encryption key\nis part of the metadata and is only available to the peers that you chose to\nsynchronize with. Think of each brig repository only as a cache for the whole\nnetwork it is in.\n\nPartial synchronisation\n~~~~~~~~~~~~~~~~~~~~~~~\n\nSometimes you only want to share certain things with certain people. You\nprobably want to share all your ``/photos`` directory with your significant\nother, but not with your fellow students. On the other hand you maybe want to\nshare the ``/lectures`` folder with them. In ``brig`` you can define what\nfolder you want to share with what remote. If you do not limit this, **all\nfolders will be open to a remote by default.** Also note, that if a remote\nalready got some content of a folder you did not want to share, he will still\nbe able to access it. If you're unsure, you should better be restrictive than\ntoo permissive.\n\nTo add a folder for a specific remote, you can use the ``folders`` subcommand\nof ``brig remote``:\n\n.. code-block:: bash\n\n # Starting with next sync, bob will only see the /videos folder:\n $ brig remote folder add bob /videos\n $ brig remote folder ls bob\n /videos\n\nIf you're tired of typing all of this, be reminded that there are very short\naliases for most subcommands:\n\n.. code-block:: bash\n\n $ brig rmt f a bob /videos\n\nIn some cases you might not trust your peers with some folders or don't want to\nhave modifications in that specific folder. For this case, ``brig`` supports\nadding a folder as ``--read-only``. Other remotes still will have access to the\nfolder, but whenever we sync with them the changes they made are ignored.\nYou can add a read-only folder by adding the ``--read-only`` switch to the command above:\n\n.. code-block:: bash\n\n $ brig rmt f a bob /videos --read-only\n\n.. note::\n\n If you want to overwrite an existing folder with new settings, you can use the ``set`` subcommand:\n\n .. code-block:: bash\n\n $ brig remote folder set bob /videos -c embrace --read-only\n\n See below for explanation on those additional options.\n\nConflicts\n~~~~~~~~~\n\nWhenever two repositories have a file at the same path, ``brig`` needs to do some conflict resolving.\nIf those files are equal or if they share common history and did not diverge there is nothing to fear.\nBut what if both sides have different versions of a file without common history? In this case ``brig`` offers you\nto handle conflict by one of the three strategies:\n\n* ``ignore``: Ignore the change from the remote side.\n* ``embrace``: Ignore our state and take over the remote's change.\n* ``marker``: Create a conflict file with the same name but a ``.conflict`` ending.\n Leave it to the user to resolve the conflict. This is the **default.**\n\nYou can configure this behavior by using ``brig cfg``:\n\n.. code-block:: bash\n\n $ brig cfg set fs.sync.conflict_strategy marker\n\nIn some cases this might not be enough though. Sometimes you might want to say\n»I trust this remote, always accept their changes«. You can do this by setting\nthe conflict strategy per remote. If no specific conflict strategy is set,\n``fs.sync.conflict_strategy`` is used. You can set the strategy by using a subcommand\nof the ``brig remote`` family:\n\n.. code-block:: bash\n\n # Always take the versions of bob on conflicts:\n $ brig remote conflict-strategy embrace bob\n\nStill not enough? You can also set the conflict strategy per folder.\nThis will trump the per-remote folder strategy:\n\n.. code-block:: bash\n\n # Use the default in all folders but use \"embrace\" in this one:\n $ brig remote folder add bob /collab -c embrace\n\nAutomatic Updating\n~~~~~~~~~~~~~~~~~~\n\n.. warning::\n\n This feature is experimental and builds upon the also experimental\n pubsub experiment of the IPFS project. Use with care.\n\n\nIf you do not want to hit ``brig sync`` every time somebody in the network changed something,\nyou can enable the automatic updating for any remote you like. Let's suppose we are ``ali``\nand want to receive updates on every change of ``bob``, we should simply add the following:\n\n.. code-block:: bash\n\n $ brig remote auto-update enable bob\n\n # (You can also abbreviate most of that:)\n # brig rmt au e bob\n\n\nAlternatively, we could have used the ``-a`` switch when adding ``bob`` as remote:\n\n.. code-block:: bash\n\n $ brig remote add bob -a\n\nIn any case, an initial sync is performed with this remote and a sync on every change\nthat ``bob`` published. Keep in mind that ``bob`` will not receive your updates by default,\nhe needs to decide to use auto updating for himself. You can watch the times when your repository\nwas updated automatically by looking at ``brig log``:\n\n\n.. code-block:: bash\n\n $ brig log\n - Sun Dec 16 18:24:27 CET 2018 • (curr)\n W1kGKKviWCBY Sun Dec 16 18:24:27 CET 2018 sync due to notification from »bob« (head)\n ...\n\nPushing changes\n~~~~~~~~~~~~~~~\n\nAs you saw above, doing a ``brig sync`` won't do a bidirectional\nsynchronisation. It will only fetch metadata from the remote and modify our\nlocal state with it. In some cases you might want to push data to a remote\n- especially when it is on one of your machines and you use for example as\narchival repository. By default pushing to a remote is rejected. You can enable\nit on a per-remote basis with this command out of the ``brig remote`` family of\ncommands:\n\n.. code-block:: bash\n\n # Allow bob and charlie to auto push to us.\n $ brig remote auto-push enable bob charlie\n\n\nNow either ``bob`` or ``charlie`` can do this from their machines:\n\n\n.. code-block:: bash\n\n # bob's machine:\n $ brig push ali\n\n\nThis will simply ask ``ali`` to do a sync with ``bob``.\n" }, { "path": "docs/tutorial/vcs.rst", "content": "Version control\n---------------\n\nOne key feature of ``brig`` over other synchronisation tools is the built-in\nand quite capable version control. If you already know ``git`` that's a plus\nfor this chapter since a lot of stuff will feel similar. This isn't a big\nsurprise, since ``brig`` implements something like ``git`` internally. Don't\nworry, knowing ``git`` is however not needed at all for this chapter.\n\nKey concepts\n~~~~~~~~~~~~\n\nI'd like you to keep the following mantra in your head when thinking\nabout versioning (repeating before you go to sleep may or may not help):\n\n**Metadata and actual data are separated.** This means that a repository may\ncontain metadata about many files, including older versions of them. However,\nit is not guaranteed that a repository caches all actual data for each file or\nversion. This is solely controlled by pinning described in the\n:ref:`pinning-section` section. If you check out earlier versions of a file,\nyou're always able to see the metadata of it, but being able to view the actual\ndata depends on having a peer that is being able to deliver the data in your\nnetwork (which might be yourself). So in short: ``brig`` **only versions\nmetadata and links to the respective data for each version**.\n\nThis is a somewhat novel approach to versioning, so feel free to re-read the\nlast paragraph, since we've found that it does not quite fit what most people\nare used to. Together with pinning this offers a high degree of freedom on how\nyou can decide what repositories store what data. The price is that this\nfine-tuned control can get a little annoying. Future versions of ``brig`` will\ntry to solve that.\n\nFor some more background, you can invoke ``brig info`` to see what metadata is\nbeing saved per file version:\n\n.. code-block:: bash\n\n $ brig show README.md\n Path /README.md\n User ali\n Type file\n Size 832 bytes\n Inode 4\n Pinned yes\n Explicit no\n ModTime 2018-10-14T22:46:00+02:00\n Tree Hash W1gX8NMQ9m8SBnjHRGtamRAjJewbnSgi6C1P7YEunfgTA3\n Content Hash W1pzHcGbVpXaePa1XpehW4HGPatDUJs8zZzSRbpNCGbN2u\n Backend Hash QmPvNjR1h56EFK1Sfb7vr7tFJ57A4JDJS9zwn7PeNbHCsK\n\n\nMost of it should be no big surprise. It might be a small surprise that three\nhashes are stored per file. The ``Backend Hash`` is really the link to the\nactual data. If you'd type ``ipfs cat\nQmPvNjR1h56EFK1Sfb7vr7tFJ57A4JDJS9zwn7PeNbHCsK`` you will get the encrypted\nversion of your file dumped to your terminal. The ``Content Hash`` is being\ncalculated before the encryption and is the same for two files with the same\ncontent. The ``Tree Hash`` is a hash that uniquely identifies this specific\nnode for internal purposes. The ``Inode`` is a number that stays unique over\nthe lifetime of a file (including moves and removes). It is used mostly in the\nFUSE filesystem.\n\nCommits\n~~~~~~~\n\nNow that we know that only metadata is versioned, we have to ask »what is the\nsmallest unit of modification that can be saved?«. This smallest unit is\na commit. A commit can be seen as a snapshot of the whole repository.\n\nThe command ``brig log`` shows you a list of commits that were made already:\n\n.. code-block:: bash\n\n - Sun Oct 14 22:46:00 CEST 2018 • (curr)\n W1kAySD3aKLt Sun Oct 14 22:46:00 CEST 2018 user: Added ali-file (head)\n W1ocyBsS28SD Sun Oct 14 22:46:00 CEST 2018 user: Added initial README.md\n W1D9KsLNnAv4 Sun Oct 14 22:46:00 CEST 2018 initial commit (init)\n\n\nEach commit is identified by a hash (e.g. ``W1kAySD3aKLt``) and records the\ntime when it was created. Apart from that, there is a message that describes\nthe commit in some way. In contrast to ``git``, **commits are rarely done by\nthe user themselve**. More often they are done by ``brig`` when synchronizing.\n\nAll commits form a long chain (**no branches**, just a linear chain) with the\nvery first empty commit called ``init`` and the still unfinished commit called\n``curr``. Directly below ``curr`` there is the last finished commit called ``head``.\n\n.. note::\n\n ``curr`` is what ``git`` users would call the staging area. While the staging area\n in ``git`` is \"special\", the ``curr`` commit can be used like any other one, with\n the sole difference that it does not have a proper hash yet.\n\nSometimes you might want to do a snapshot or »savepoint« yourself. In this case\nyou can do a commit yourself:\n\n.. code-block:: bash\n\n $ brig touch A_NEW_FILE\n $ brig commit -m 'better leave some breadcrumbs'\n $ brig log | head -n 2\n - Mon Oct 15 00:27:37 CEST 2018 • (curr)\n W1hZoY7TrxyK Sun Oct 14 22:46:00 CEST 2018 user: better leave some bread crumbs (head)\n\nThis snapshot can be useful later if you decide to revert to a certain version.\nThe hash of the commit is of course hard to remember, so if you need it very often, you can\ngive it a tag yourself. Tags are similar to the names, ``curr``, ``head`` and ``init`` but\nwon't be changed by ``brig`` and won't move therefore:\n\n.. code-block:: bash\n\n # instead of \"W1hZoY7TrxyK\" you also could use \"head\" here:\n $ brig tag W1hZoY7TrxyK breadcrumbs\n $ brig log | grep breadcrumbs\n $ W1hZoY7TrxyK Sun Oct 14 22:46:00 CEST 2018 user: better leave some bread crumbs (breadcrumbs, head)\n\n\nFile history\n~~~~~~~~~~~~\n\nEach file and directory in ``brig`` maintains its own history. Each entry of\nthis history relates to exactly one distinct commit. In the life of a file or\ndirectory there are four things that can happen to it:\n\n- *added:* The file was added in this commit.\n- *moved:* The file was moved in this commit.\n- *removed:* The file was removed in this commit.\n- *modified:* The file's content (i.e. hash changed) was altered in this commit.\n\nYou can check an individual file or directorie's history by using the ``brig history`` command:\n\n.. code-block:: bash\n\n # or \"hst\" for short:\n $ brig hst README.md\n CHANGE FROM TO WHEN\n added INIT W1ocyBsS28SD Oct 14 22:46:00\n $ brig mv README.md README_LATER.md\n $ brig hst README_LATER.md\n CHANGE FROM TO HOW WHEN\n moved HEAD CURR /README.md → /README_LATER.md Oct 15 00:27:37\n added INIT W1ocyBsS28SD Oct 14 22:46:0\n\nAs you can see, you will be shown one line per history entry. Each entry\ndenotes which commit the change was in. Some commits were nothing was changed\nwill be jumped over except if you pass ``--empty``.\n\nViewing differences\n~~~~~~~~~~~~~~~~~~~\n\nIf you're interested what changed in a range of your own commits, you can use\nthe ``brig diff`` command as shown previously. The ``-s`` (``--self``) switch\nsays that we want to compare only two of our own commits (as opposed to\ncomparing with the commits of a remote).\n\n.. code-block:: bash\n\n # Let's compare the commit hashes from above:\n $ brig diff -s W1hZoY7TrxyK W1kAySD3aKLt\n •\n └── + A_NEW_FILE\n\nOften, those hashes are quite hard to remember and annoying to look up. That's\nwhy you can the special syntax ``^`` to denote that you want to go\n»one commit up«:\n\n.. code-block:: bash\n\n brig diff -s head head^\n •\n └── + A_NEW_FILE\n # You can also use this several times:\n brig diff -s head^^^ head^^^^^\n •\n └── + README.md\n\nIf you just want to see what you changed since ``head``, you can simply type ``brig diff``.\nThis is the same as ``brig diff -s curr head``:\n\n.. code-block:: bash\n\n $ brig diff\n •\n └── README.md → README_LATER.md\n $ brig diff -s curr head\n •\n └── README.md → README_LATER.md\n\n\nReverting to previous state\n~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nUntil now we were only looking at the version history and didn't modify it. The\nmost versatile command to do that is ``brig reset``. It is able to revert\nchanges previously made:\n\n.. code-block:: bash\n\n # Reset to the \"init\" commit (the very first and empty commit)\n $ brig reset init\n $ brig ls # nothing, it's empty.\n\n\nThe key here is that you did not loose any history:\n\n.. code-block:: bash\n\n $ brig log | head -2\n - Mon Oct 15 00:51:12 CEST 2018 • (curr)\n W1hZoY7TrxyK Sun Oct 14 22:46:00 CEST 2018 user: better leave some bread crumbs (breadcrumbs)\n\n\nAs you can see, we still have the previous commits. ``brig revert`` did one\nthing more than restoring the state of ``init`` and put that result in\n``curr``. This also means that you can't really *modify* history. But you can\nrevert it. Let's revert your complete wipe-out:\n\n.. code-block:: bash\n\n # Reset to the state we had in »breadcrumbs«\n $ brig reset breadcrumbs\n\n\n``brig reset`` cannot only restore old commits, but individual files and\ndirectories:\n\n.. code-block:: bash\n\n $ brig reset head^^ README.md\n\n.. note::\n\n It is a good idea to do a ``brig commit`` before a ``brig reset``. Since it\n modifies ``curr`` you might loose uncommitted changes. It will warn you\n about that, but you can overwrite that warning with ``--force``. If you did\n a ``brig commit`` you can simply use ``brig reset head`` to go back to the\n last good state.\n" }, { "path": "events/backend/backend.go", "content": "package backend\n\nimport (\n\t\"context\"\n\t\"io\"\n)\n\n// Message is returned by Subscribe.\n// It encapsulates a single event message coming\n// from another remote.\ntype Message interface {\n\t// Data is the data that is sent alongside the message.\n\tData() []byte\n\t// Source is the addr of the remote.\n\tSource() string\n}\n\n// Subscription is an iterator like interface for accessing and listening\n// for messages from other remotes.\ntype Subscription interface {\n\tio.Closer\n\n\t// Next blocks until receiving a new message or fails with\n\t// context.Canceled if the cancel func was called.\n\tNext(ctx context.Context) (Message, error)\n}\n\n// Backend is the backend that backends of the event subsystem must fulfill.\ntype Backend interface {\n\t// Subscribe returns a new Subscription iterator for `topic`.\n\tSubscribe(ctx context.Context, topic string) (Subscription, error)\n\t// PublishEvent sends `data` to all listening remotes on `topic`.\n\tPublishEvent(topic string, data []byte) error\n}\n" }, { "path": "events/capnp/events_api.capnp", "content": "using Go = import \"/go.capnp\";\n\n@0xfc8938b535319bfe;\n$Go.package(\"capnp\");\n$Go.import(\"github.com/sahib/brig/events/capnp\");\n\nstruct Event $Go.doc(\"\") {\n type @0 :Text;\n}\n" }, { "path": "events/capnp/events_api.capnp.go", "content": "// Code generated by capnpc-go. DO NOT EDIT.\n\npackage capnp\n\nimport (\n\tcapnp \"zombiezen.com/go/capnproto2\"\n\ttext \"zombiezen.com/go/capnproto2/encoding/text\"\n\tschemas \"zombiezen.com/go/capnproto2/schemas\"\n)\n\ntype Event struct{ capnp.Struct }\n\n// Event_TypeID is the unique identifier for the type Event.\nconst Event_TypeID = 0x9c032508b61d1d09\n\nfunc NewEvent(s *capnp.Segment) (Event, error) {\n\tst, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 1})\n\treturn Event{st}, err\n}\n\nfunc NewRootEvent(s *capnp.Segment) (Event, error) {\n\tst, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 1})\n\treturn Event{st}, err\n}\n\nfunc ReadRootEvent(msg *capnp.Message) (Event, error) {\n\troot, err := msg.RootPtr()\n\treturn Event{root.Struct()}, err\n}\n\nfunc (s Event) String() string {\n\tstr, _ := text.Marshal(0x9c032508b61d1d09, s.Struct)\n\treturn str\n}\n\nfunc (s Event) Type() (string, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.Text(), err\n}\n\nfunc (s Event) HasType() bool {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s Event) TypeBytes() ([]byte, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.TextBytes(), err\n}\n\nfunc (s Event) SetType(v string) error {\n\treturn s.Struct.SetText(0, v)\n}\n\n// Event_List is a list of Event.\ntype Event_List struct{ capnp.List }\n\n// NewEvent creates a new list of Event.\nfunc NewEvent_List(s *capnp.Segment, sz int32) (Event_List, error) {\n\tl, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 0, PointerCount: 1}, sz)\n\treturn Event_List{l}, err\n}\n\nfunc (s Event_List) At(i int) Event { return Event{s.List.Struct(i)} }\n\nfunc (s Event_List) Set(i int, v Event) error { return s.List.SetStruct(i, v.Struct) }\n\nfunc (s Event_List) String() string {\n\tstr, _ := text.MarshalList(0x9c032508b61d1d09, s.List)\n\treturn str\n}\n\n// Event_Promise is a wrapper for a Event promised by a client call.\ntype Event_Promise struct{ *capnp.Pipeline }\n\nfunc (p Event_Promise) Struct() (Event, error) {\n\ts, err := p.Pipeline.Struct()\n\treturn Event{s}, err\n}\n\nconst schema_fc8938b535319bfe = \"x\\xda\\x12\\xd0s`\\x12d\\x8dg`\\x08dae\\xfb\" +\n\t\"\\xcf)+\\xbb\\x8dC\\x95y\\x0e\\x83\\xa0\\x1c\\xe3\\xff\\x7f\\xb3\" +\n\t\"\\x0dM\\xb7Zt\\xfea`edg`0\\x14Ub\" +\n\t\"\\x14Te\\x17T\\x95\\x17t\\xb5g`\\xfc\\x9fZ\\x96\\x9a\" +\n\t\"WR\\xac\\x9f\\xcc\\x9cX\\x90W\\xa0\\x0f\\xe1\\xc5'\\x16d\" +\n\t\"\\xea%\\x83\\x04\\xac\\\\\\xcb\\xd8S\\xf3J\\x02\\x18\\x19\\x03Y\" +\n\t\"\\x18\\x99\\xfe\\xc7M\\x9e\\x1f\\xb8\\xf7Z\\xd7Q\\x86@\\x16&\" +\n\t\"FG\\x01FF\\x1e\\x06\\x06AF.\\x06\\xc6@\\x16f\" +\n\t\"\\x16\\x06\\x06\\x16F\\x06\\x06A^-\\x06\\x86@\\x0ef\\xc6\" +\n\t\"@\\x11&F\\xfe\\x92\\xca\\x82TF\\x1e\\x06&F\\x1e\\x06\" +\n\t\"F@\\x00\\x00\\x00\\xff\\xff\\x11'$\\xac\"\n\nfunc init() {\n\tschemas.Register(schema_fc8938b535319bfe,\n\t\t0x9c032508b61d1d09)\n}\n" }, { "path": "events/docs.go", "content": "// Package events is the event notification subsystem of brig.\n// It uses the backend's capabilities (in case of IPFS we use pubsub)\n// to publish and subscribe to a topic of events. If an event was received\n// it is forwarded to the caller side in order to react on it.\npackage events\n" }, { "path": "events/event.go", "content": "package events\n\nimport (\n\t\"fmt\"\n\n\tcapnp_model \"github.com/sahib/brig/events/capnp\"\n\tcapnp \"zombiezen.com/go/capnproto2\"\n)\n\nconst (\n\t// UnknownEvent should not happen in practice.\n\tUnknownEvent = EventType(1 << iota)\n\t// FsEvent tells other remotes that our filesystem changed.\n\tFsEvent\n\t// NetEvent indicates to other peers that our network status changed.\n\tNetEvent\n)\n\n// EventType is the type of a\ntype EventType int\n\n// String returns a human readable representation of the event type\nfunc (ev EventType) String() string {\n\tswitch ev {\n\tcase FsEvent:\n\t\treturn \"fs\"\n\tcase NetEvent:\n\t\treturn \"net\"\n\tdefault:\n\t\treturn \"unknown\"\n\t}\n}\n\n// EventFromString tries to parse `ev` as event type.\n// If it fails, an error will be returned.\nfunc EventFromString(ev string) (EventType, error) {\n\tswitch ev {\n\tcase \"fs\":\n\t\treturn FsEvent, nil\n\tcase \"net\":\n\t\treturn NetEvent, nil\n\tdefault:\n\t\treturn UnknownEvent, fmt.Errorf(\"unknown EventType type: %s\", ev)\n\t}\n}\n\n// Event is a event that can be published or received by the event subsystem.\ntype Event struct {\n\tType EventType\n\tSource string\n}\n\nfunc (msg *Event) encode() ([]byte, error) {\n\tcapMsg, seg, err := capnp.NewMessage(capnp.SingleSegment(nil))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapEv, err := capnp_model.NewRootEvent(seg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := capEv.SetType(msg.Type.String()); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn capMsg.Marshal()\n}\n\nfunc decodeMessage(data []byte) (*Event, error) {\n\tif data == nil {\n\t\treturn nil, fmt.Errorf(\"received empty event\")\n\t}\n\n\tcapMsg, err := capnp.Unmarshal(data)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapEv, err := capnp_model.ReadRootEvent(capMsg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapEvType, err := capEv.Type()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tev, err := EventFromString(capEvType)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &Event{Type: ev}, nil\n}\n\nfunc dedupeEvents(evs []Event) []Event {\n\tseen := make(map[EventType]map[string]bool)\n\tdedupEvs := []Event{}\n\n\tfor _, ev := range evs {\n\t\tseenSources, ok := seen[ev.Type]\n\t\tif ok {\n\t\t\tif seenSources[ev.Source] {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t} else {\n\t\t\tseenSources = make(map[string]bool)\n\t\t\tseen[ev.Type] = seenSources\n\t\t}\n\n\t\tdedupEvs = append(dedupEvs, ev)\n\t\tseen[ev.Type][ev.Source] = true\n\t}\n\n\treturn dedupEvs\n}\n" }, { "path": "events/listener.go", "content": "package events\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/sahib/brig/events/backend\"\n\t\"github.com/sahib/config\"\n\tlog \"github.com/sirupsen/logrus\"\n\t\"golang.org/x/time/rate\"\n)\n\nconst (\n\tbrigEventTopicPrefix = \"brig/events/\"\n\tmaxBurstSize = 100\n)\n\n// Listener listens to incoming events from other remotes.\n// For every event, a registered callback can be executed.\n// It does not implement net.Listener and is only similar from a concept POV.\ntype Listener struct {\n\tmu sync.Mutex\n\n\tbk backend.Backend\n\tcfg *config.Config\n\tcallbacks map[EventType][]callback\n\tcancels map[string]context.CancelFunc\n\tevSendCh chan Event\n\tevRecvCh chan Event\n\townAddr string\n\tisClosed bool\n}\n\ntype callback struct {\n\tfn func(*Event)\n\tnotifyOnOwn bool\n}\n\n// NewListener constructs a new listener.\n// `cfg` is used to read the event subsystem config.\n// `bk` is a events.Backend.\n// `ownAddr` is the addr of our own node.\nfunc NewListener(cfg *config.Config, bk backend.Backend, ownAddr string) *Listener {\n\tlst := &Listener{\n\t\tbk: bk,\n\t\tcfg: cfg,\n\t\townAddr: ownAddr,\n\t\tcallbacks: make(map[EventType][]callback),\n\t\tcancels: make(map[string]context.CancelFunc),\n\t\tevSendCh: make(chan Event, maxBurstSize),\n\t\tevRecvCh: make(chan Event, maxBurstSize),\n\t}\n\n\tgo lst.eventSendLoop()\n\tgo lst.eventRecvLoop()\n\treturn lst\n}\n\n// Close will close all open listeners and clean up internal resources.\nfunc (lst *Listener) Close() error {\n\tlst.mu.Lock()\n\tdefer lst.mu.Unlock()\n\n\tif lst.isClosed {\n\t\treturn nil\n\t}\n\n\tclose(lst.evSendCh)\n\tclose(lst.evRecvCh)\n\n\tfor _, cancel := range lst.cancels {\n\t\tcancel()\n\t}\n\n\tlst.isClosed = true\n\treturn nil\n}\n\n// RegisterEventHandler remembers that `hdl` should be called whenever a event\n// of type `ev` is being received. If `notifyOnOwn` is true, the handler\n// will only be called for changes that came from our own node. If it is `false`\n// it will only be called for\nfunc (lst *Listener) RegisterEventHandler(ev EventType, notifyOnOwn bool, hdl func(ev *Event)) {\n\tlst.mu.Lock()\n\tdefer lst.mu.Unlock()\n\n\tif lst.isClosed {\n\t\treturn\n\t}\n\n\tlst.callbacks[ev] = append(lst.callbacks[ev], callback{\n\t\tfn: hdl,\n\t\tnotifyOnOwn: notifyOnOwn,\n\t})\n}\n\nfunc eventLoop(evCh chan Event, interval time.Duration, rps float64, fn func(ev Event)) {\n\ttckr := time.NewTicker(interval)\n\tdefer tckr.Stop()\n\n\t// Use a time window approach to dedupe incoming events\n\t// and to process them in a batch (in order to avoid work)\n\t// We still rate limit while processing too many at the same time.\n\tevents := []Event{}\n\tlim := rate.NewLimiter(rate.Limit(rps), maxBurstSize)\n\n\tfor {\n\t\tselect {\n\t\tcase <-tckr.C:\n\t\t\t// Flush phase. Deduple all events and send them out to the handler\n\t\t\t// in a possibly time throttled manner.\n\t\t\tevents = dedupeEvents(events)\n\t\t\tif len(events) == 0 {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\t// Apply the rate limiting only after\n\t\t\tr := lim.ReserveN(time.Now(), len(events))\n\t\t\tif !r.OK() {\n\t\t\t\t// would only happen if the burst size is too big.\n\t\t\t\t// drop all events in this special case.\n\t\t\t\tevents = []Event{}\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tdelay := r.Delay()\n\t\t\tfor _, ev := range events {\n\t\t\t\tfn(ev)\n\n\t\t\t\t// spread the work over the processing of all events:\n\t\t\t\ttime.Sleep(delay / time.Duration(len(events)))\n\t\t\t}\n\n\t\t\tevents = []Event{}\n\t\tcase ev, ok := <-evCh:\n\t\t\tif !ok {\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tif len(events) > maxBurstSize {\n\t\t\t\t// drop events if the list gets too big:\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tevents = append(events, ev)\n\t\t}\n\t}\n}\n\nfunc (lst *Listener) eventRecvLoop() {\n\trecvInterval := lst.cfg.Duration(\"recv_interval\")\n\trecvMaxEvRPS := lst.cfg.Float(\"recv_max_events_per_second\")\n\n\teventLoop(lst.evRecvCh, recvInterval, recvMaxEvRPS, func(ev Event) {\n\t\tlst.mu.Lock()\n\t\tif cbs, ok := lst.callbacks[ev.Type]; ok {\n\t\t\tfor _, cb := range cbs {\n\t\t\t\tif !cb.notifyOnOwn {\n\t\t\t\t\tgo cb.fn(&ev)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\tlst.mu.Unlock()\n\t})\n}\n\nfunc (lst *Listener) eventSendLoop() {\n\townTopic := brigEventTopicPrefix + lst.ownAddr\n\n\tsendInterval := lst.cfg.Duration(\"send_interval\")\n\tsendMaxEvRPS := lst.cfg.Float(\"send_max_events_per_second\")\n\n\teventLoop(lst.evSendCh, sendInterval, sendMaxEvRPS, func(ev Event) {\n\t\tdata, err := ev.encode()\n\t\tif err != nil {\n\t\t\tlog.Errorf(\"event: failed to encode: %v\", err)\n\t\t\treturn\n\t\t}\n\n\t\tif err := lst.bk.PublishEvent(ownTopic, data); err != nil {\n\t\t\tlog.Errorf(\"event: failed to publish: %v\", err)\n\t\t\treturn\n\t\t}\n\t})\n}\n\nfunc (lst *Listener) publishToSelf(ev Event) {\n\tif cbs, ok := lst.callbacks[ev.Type]; ok {\n\t\tfor _, cb := range cbs {\n\t\t\tif cb.notifyOnOwn {\n\t\t\t\tgo cb.fn(&ev)\n\t\t\t}\n\t\t}\n\t}\n}\n\n// PublishEvent notifies other peers that something on our\n// side changed. The \"something\" is defined by `ev`.\n// PublishEvent does not block.\nfunc (lst *Listener) PublishEvent(ev Event) error {\n\tlst.mu.Lock()\n\tdefer lst.mu.Unlock()\n\n\tif lst.isClosed {\n\t\treturn nil\n\t}\n\n\tif !lst.cfg.Bool(\"enabled\") {\n\t\treturn nil\n\t}\n\n\t// Some submodules (like the gateway) also want to be notified\n\t// when other parts of the same server (fuse, cmdline) changed something.\n\tlst.publishToSelf(ev)\n\n\t// Only send the event if we are not clogged up yet.\n\t// We prioritze the well-being of other systems more by\n\t// not allowing PublishEvent to block.\n\tselect {\n\tcase lst.evSendCh <- ev:\n\t\treturn nil\n\tdefault:\n\t\treturn fmt.Errorf(\"lost event: %v\", ev)\n\t}\n}\n\n// SetupListeners sets up the listener to receive events from any of `addrs`.\n// If `ctx` is being canceled, all listeners will stop.\n// SetupListeners can be called several times, each time overwriting and stopping\n// previous listeners.\nfunc (lst *Listener) SetupListeners(ctx context.Context, addrs []string) error {\n\tif lst.isClosed {\n\t\treturn nil\n\t}\n\n\tseen := make(map[string]bool)\n\n\tfor _, addr := range addrs {\n\t\tseen[addr] = true\n\t\tcancel, ok := lst.cancels[addr]\n\t\tif ok {\n\t\t\t// We already have a listener for this.\n\t\t\tcontinue\n\t\t}\n\n\t\tctx, cancel := context.WithCancel(ctx)\n\t\tlst.cancels[addr] = cancel\n\t\tgo lst.listenSingle(ctx, brigEventTopicPrefix+addr)\n\t}\n\n\t// cancel all listeners that are not needed anymore.\n\tfor addr, cancel := range lst.cancels {\n\t\tif !seen[addr] {\n\t\t\tcancel()\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (lst *Listener) listenSingle(ctx context.Context, topic string) error {\n\tsub, err := lst.bk.Subscribe(ctx, topic)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdefer sub.Close()\n\n\tlog.Debugf(\"listening for events on %s\", topic)\n\tdefer log.Debugf(\"event listener on %s closing\", topic)\n\n\tfor {\n\t\tif !lst.cfg.Bool(\"enabled\") {\n\t\t\t// Do not grind the cpu if it is not enabled.\n\t\t\ttime.Sleep(2 * time.Second)\n\t\t\tcontinue\n\t\t}\n\n\t\tmsg, err := sub.Next(ctx)\n\t\tif msg == nil {\n\t\t\t// Sometimes we might have a case where a ipfs daemon\n\t\t\t// returns an empty message very often - just sleep a bit\n\t\t\t// to save the cpu.\n\t\t\ttime.Sleep(500 * time.Millisecond)\n\t\t\tcontinue\n\t\t}\n\n\t\tif err == io.EOF || err == context.Canceled {\n\t\t\treturn nil\n\t\t} else if err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif msg.Source() == lst.ownAddr {\n\t\t\tcontinue\n\t\t}\n\n\t\tev, err := decodeMessage(msg.Data())\n\t\tif err != nil {\n\t\t\tlog.Warningf(\"received bad message: %v\", err)\n\t\t\tcontinue\n\t\t}\n\n\t\tev.Source = msg.Source()\n\n\t\tif lst.isClosed {\n\t\t\tbreak\n\t\t}\n\n\t\tselect {\n\t\tcase lst.evRecvCh <- *ev:\n\t\tdefault:\n\t\t\tlog.Warningf(\"dropped incoming event: %v\", ev)\n\t\t}\n\t}\n\n\treturn nil\n}\n" }, { "path": "events/listener_test.go", "content": "package events\n\nimport (\n\t\"context\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/sahib/brig/defaults\"\n\t\"github.com/sahib/brig/events/mock\"\n\t\"github.com/sahib/config\"\n\tlog \"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc withEventListener(t *testing.T, ownAddr string, fn func(lst *Listener)) {\n\tcfg, err := config.Open(nil, defaults.Defaults, config.StrictnessPanic)\n\trequire.Nil(t, err)\n\n\tcfg.SetDuration(\"events.recv_interval\", time.Millisecond*1)\n\tcfg.SetDuration(\"events.send_interval\", time.Millisecond*1)\n\n\tcfg.SetFloat(\"events.recv_max_events_per_second\", 0.1)\n\tcfg.SetFloat(\"events.send_max_events_per_second\", 0.1)\n\n\tevb := mock.NewEventsBackend(ownAddr)\n\tlst := NewListener(cfg.Section(\"events\"), evb, ownAddr)\n\tfn(lst)\n\trequire.Nil(t, lst.Close())\n}\n\nfunc withEventListenerPair(t *testing.T, addrA, addrB string, fn func(lstA, lstB *Listener)) {\n\twithEventListener(t, addrA, func(lstA *Listener) {\n\t\twithEventListener(t, addrB, func(lstB *Listener) {\n\t\t\tfn(lstA, lstB)\n\t\t})\n\t})\n}\n\nfunc TestBasicRun(t *testing.T) {\n\tlog.SetLevel(log.DebugLevel)\n\n\twithEventListenerPair(t, \"a\", \"b\", func(lstA, lstB *Listener) {\n\t\teventReceived := false\n\n\t\tlstB.RegisterEventHandler(FsEvent, false, func(ev *Event) {\n\t\t\trequire.Equal(t, \"a\", ev.Source)\n\t\t\trequire.Equal(t, FsEvent, ev.Type)\n\t\t\teventReceived = true\n\t\t})\n\n\t\trequire.Nil(t, lstB.SetupListeners(context.Background(), []string{\"a\"}))\n\n\t\tfor i := 0; i < 100; i++ {\n\t\t\trequire.Nil(t, lstA.PublishEvent(Event{Type: FsEvent}))\n\t\t}\n\n\t\ttime.Sleep(500 * time.Millisecond)\n\t\trequire.True(t, eventReceived)\n\n\t\t// Do a double close:\n\t\trequire.Nil(t, lstA.Close())\n\t\trequire.Nil(t, lstA.PublishEvent(Event{Type: NetEvent}))\n\t\ttime.Sleep(200 * time.Millisecond)\n\t})\n}\n" }, { "path": "events/mock/mock.go", "content": "package mock\n\nimport (\n\t\"context\"\n\t\"sync\"\n\n\teventsBackend \"github.com/sahib/brig/events/backend\"\n)\n\nvar subs map[string][]*mockSubscription\nvar subsLock sync.Mutex\n\nfunc init() {\n\tsubs = make(map[string][]*mockSubscription)\n}\n\n// EventsBackend fakes the event backend by setting up a very basic\n// message broker in memory and tunneling all messages over it.\ntype EventsBackend struct {\n\townAddr string\n}\n\n// NewEventsBackend returns a new EventsBackend\nfunc NewEventsBackend(ownAddr string) *EventsBackend {\n\treturn &EventsBackend{\n\t\townAddr: ownAddr,\n\t}\n}\n\ntype mockMessage struct {\n\tdata []byte\n\tsource string\n}\n\nfunc (mm mockMessage) Data() []byte {\n\treturn mm.data\n}\n\nfunc (mm mockMessage) Source() string {\n\treturn mm.source\n}\n\ntype mockSubscription struct {\n\tmsgs chan mockMessage\n}\n\nfunc (ms *mockSubscription) Next(ctx context.Context) (eventsBackend.Message, error) {\n\tselect {\n\tcase <-ctx.Done():\n\t\treturn nil, ctx.Err()\n\tcase msg := <-ms.msgs:\n\t\treturn msg, nil\n\t}\n}\n\nfunc (ms *mockSubscription) Close() error {\n\treturn nil\n}\n\n// Subscribe is a mock implementation meant for testing.\nfunc (mb *EventsBackend) Subscribe(ctx context.Context, topic string) (eventsBackend.Subscription, error) {\n\tsubsLock.Lock()\n\tdefer subsLock.Unlock()\n\n\tnewSub := &mockSubscription{\n\t\tmsgs: make(chan mockMessage, 100),\n\t}\n\n\tsubs[topic] = append(subs[topic], newSub)\n\treturn newSub, nil\n}\n\n// PublishEvent is a mock implementation meant for testing.\nfunc (mb *EventsBackend) PublishEvent(topic string, data []byte) error {\n\tsubsLock.Lock()\n\tdefer subsLock.Unlock()\n\n\tsubs, ok := subs[topic]\n\tif !ok {\n\t\treturn nil\n\t}\n\n\tfor _, sub := range subs {\n\t\tdataCopy := make([]byte, len(data))\n\t\tcopy(dataCopy, data)\n\n\t\tsub.msgs <- mockMessage{\n\t\t\tdata: dataCopy,\n\t\t\tsource: mb.ownAddr,\n\t\t}\n\t}\n\n\treturn nil\n}\n" }, { "path": "fuse/directory.go", "content": "// +build !windows\n\npackage fuse\n\nimport (\n\t\"os\"\n\t\"path\"\n\t\"time\"\n\n\t\"context\"\n\n\t\"bazil.org/fuse\"\n\t\"bazil.org/fuse/fs\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// Directory represents a directory node.\ntype Directory struct {\n\tpath string\n\tm *Mount\n}\n\n// Attr is called to retrieve stat-metadata about the directory.\nfunc (dir *Directory) Attr(ctx context.Context, attr *fuse.Attr) error {\n\tdefer logPanic(\"dir: attr\")\n\n\tdebugLog(\"Exec dir attr: %v\", dir.path)\n\tinfo, err := dir.m.fs.Stat(dir.path)\n\tif err != nil {\n\t\treturn errorize(\"dir-attr\", err)\n\t}\n\n\t// Act like the file is owned by the user of the brig process.\n\tattr.Uid = uint32(os.Getuid())\n\tattr.Gid = uint32(os.Getgid())\n\n\tattr.Mode = os.ModeDir | 0755\n\tattr.Size = info.Size\n\tattr.Mtime = info.ModTime\n\tattr.Inode = info.Inode\n\treturn nil\n}\n\n// Lookup is called to lookup a direct child of the directory.\nfunc (dir *Directory) Lookup(ctx context.Context, name string) (fs.Node, error) {\n\tdefer logPanic(\"dir: lookup\")\n\n\tdebugLog(\"Exec lookup: %v\", name)\n\tif name == \".\" {\n\t\treturn dir, nil\n\t}\n\n\tif name == \"..\" && dir.path != \"/\" {\n\t\treturn &Directory{path: path.Dir(dir.path), m: dir.m}, nil\n\t}\n\n\tvar result fs.Node\n\tchildPath := path.Join(dir.path, name)\n\n\tinfo, err := dir.m.fs.Stat(childPath)\n\tif err != nil {\n\t\treturn nil, errorize(\"dir-lookup\", err)\n\t}\n\n\tif info.IsDir {\n\t\tresult = &Directory{path: childPath, m: dir.m}\n\t} else {\n\t\tresult = &File{path: childPath, m: dir.m}\n\t}\n\n\treturn result, nil\n}\n\n// Mkdir is called to create a new directory node inside the receiver.\nfunc (dir *Directory) Mkdir(ctx context.Context, req *fuse.MkdirRequest) (fs.Node, error) {\n\tdefer logPanic(\"dir: mkdir\")\n\n\tdebugLog(\"fuse-mkdir: %v\", req.Name)\n\n\tchildPath := path.Join(dir.path, req.Name)\n\tif err := dir.m.fs.Mkdir(childPath, false); err != nil {\n\t\tlog.WithFields(log.Fields{\n\t\t\t\"path\": childPath,\n\t\t\t\"error\": err,\n\t\t}).Warning(\"fuse-mkdir failed\")\n\n\t\treturn nil, fuse.EIO\n\t}\n\n\tnotifyChange(dir.m, 100*time.Millisecond)\n\treturn &Directory{path: childPath, m: dir.m}, nil\n}\n\n// Create is called to create an opened file or directory as child of the receiver.\nfunc (dir *Directory) Create(ctx context.Context, req *fuse.CreateRequest, resp *fuse.CreateResponse) (fs.Node, fs.Handle, error) {\n\tdefer logPanic(\"dir: create\")\n\n\tvar err error\n\tdebugLog(\"fuse-create: %v\", req.Name)\n\n\tchildPath := path.Join(dir.path, req.Name)\n\tswitch {\n\tcase req.Mode&os.ModeDir != 0:\n\t\terr = dir.m.fs.Mkdir(childPath, false)\n\tdefault:\n\t\terr = dir.m.fs.Touch(childPath)\n\t}\n\n\tif err != nil {\n\t\tlog.WithFields(log.Fields{\n\t\t\t\"path\": childPath,\n\t\t\t\"error\": err,\n\t\t}).Warning(\"fuse-create failed\")\n\t\treturn nil, nil, fuse.EIO\n\t}\n\n\tfd, err := dir.m.fs.Open(childPath)\n\tif err != nil {\n\t\treturn nil, nil, errorize(\"fuse-dir-create\", err)\n\t}\n\n\tnotifyChange(dir.m, 100*time.Millisecond)\n\tfile := &File{path: childPath, m: dir.m}\n\treturn file, &Handle{fd: fd, m: dir.m}, nil\n}\n\n// Remove is called when a direct child in the directory needs to be removed.\nfunc (dir *Directory) Remove(ctx context.Context, req *fuse.RemoveRequest) error {\n\tdefer logPanic(\"dir: remove\")\n\n\tpath := path.Join(dir.path, req.Name)\n\tif err := dir.m.fs.Remove(path); err != nil {\n\t\tlog.Errorf(\"fuse: dir-remove: `%s` failed: %v\", path, err)\n\t\treturn fuse.ENOENT\n\t}\n\n\tnotifyChange(dir.m, 100*time.Millisecond)\n\treturn nil\n}\n\n// ReadDirAll is called to get a directory listing of the receiver.\nfunc (dir *Directory) ReadDirAll(ctx context.Context) ([]fuse.Dirent, error) {\n\tdefer logPanic(\"dir: readdirall\")\n\n\tdebugLog(\"Exec read dir all\")\n\tselfInfo, err := dir.m.fs.Stat(dir.path)\n\tif err != nil {\n\t\tlog.Debugf(\"Failed to stat: %v\", dir.path)\n\t\treturn nil, errorize(\"fuse-dir-ls-stat\", err)\n\t}\n\n\tparentDir := path.Dir(dir.path)\n\tparInfo, err := dir.m.fs.Stat(parentDir)\n\tif err != nil {\n\t\tlog.Debugf(\"Failed to stat parent: %v\", parentDir)\n\t\treturn nil, errorize(\"fuse-dir-ls-stat-par\", err)\n\t}\n\n\tfuseEnts := []fuse.Dirent{\n\t\t{\n\t\t\tInode: selfInfo.Inode,\n\t\t\tType: fuse.DT_Dir,\n\t\t\tName: \".\",\n\t\t},\n\t\t{\n\t\t\tInode: parInfo.Inode,\n\t\t\tType: fuse.DT_Dir,\n\t\t\tName: \"..\",\n\t\t},\n\t}\n\n\tentries, err := dir.m.fs.List(dir.path, 1)\n\tif err != nil {\n\t\tlog.Warningf(\"Failed to list entries: %v\", dir.path)\n\t\treturn nil, errorize(\"fuse-dir-readall\", err)\n\t}\n\n\tfor _, entry := range entries {\n\t\tchildType := fuse.DT_File\n\t\tif entry.IsDir {\n\t\t\tchildType = fuse.DT_Dir\n\t\t}\n\n\t\t// If we return the same path (or just \"/\") to fuse\n\t\t// it will return a EIO to userland. Weird.\n\t\tif entry.Path == \"/\" || entry.Path == dir.path {\n\t\t\tcontinue\n\t\t}\n\n\t\tfuseEnts = append(fuseEnts, fuse.Dirent{\n\t\t\tInode: entry.Inode,\n\t\t\tType: childType,\n\t\t\tName: path.Base(entry.Path),\n\t\t})\n\t}\n\n\treturn fuseEnts, nil\n}\n\n// Rename or move files or directories\n// TODO: fix info availability,\n// somehow the info about moved item is not visible for a little while after move\n// It usually available after a second or two.\n// How to reproduce\n// mv file1 file2\n// ls -l file2\n// You will see that username, permission, size, date, and so on all in question marks\n// For what I can see. ls cannot access this particular file, even though\n// It will appear as an entry in the call to ReadDirAll done by `ls on_dir`\n// Seems to be cache related issue\nfunc (dir *Directory) Rename(ctx context.Context, req *fuse.RenameRequest, newDir fs.Node) error {\n\tdefer logPanic(\"dir: rename\")\n\n\tdebugLog(\"exec dir rename\")\n\tnewParent, ok := newDir.(*Directory)\n\tif !ok {\n\t\treturn fuse.EIO\n\t}\n\toldPath := path.Join(dir.path, req.OldName)\n\tnewPath := path.Join(newParent.path, req.NewName)\n\tif err := dir.m.fs.Move(oldPath, newPath); err != nil {\n\t\tlog.Warningf(\"fuse: dir: mv: %v\", err)\n\t\treturn err\n\t}\n\n\tnotifyChange(dir.m, 100*time.Millisecond)\n\treturn nil\n}\n\n// Getxattr is called to get a single xattr (extended attribute) of a directory.\nfunc (dir *Directory) Getxattr(ctx context.Context, req *fuse.GetxattrRequest, resp *fuse.GetxattrResponse) error {\n\tdefer logPanic(\"dir: getxattr\")\n\n\t// Do not worry about req.Size\n\t// fuse will cut it to allowed size and report to the caller that buffer need to be larger\n\txattrs, err := getXattr(dir.m.fs, req.Name, dir.path)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tresp.Xattr = xattrs\n\treturn nil\n}\n\n// Setxattr is called by the setxattr syscall.\nfunc (dir *Directory) Setxattr(ctx context.Context, req *fuse.SetxattrRequest) error {\n\tdefer logPanic(\"dir: setxattr\")\n\n\treturn setXattr(dir.m.fs, req.Name, dir.path, req.Xattr)\n}\n\n// Listxattr is called to list all xattrs of this directory.\nfunc (dir *Directory) Listxattr(ctx context.Context, req *fuse.ListxattrRequest, resp *fuse.ListxattrResponse) error {\n\tdefer logPanic(\"dir: listxattr\")\n\n\t// Do not worry about req.Size\n\t// fuse will cut it to allowed size and report to the caller that buffer need to be larger\n\tresp.Xattr = listXattr()\n\treturn nil\n}\n\nvar _ = fs.NodeGetxattrer(&Directory{})\nvar _ = fs.NodeListxattrer(&Directory{})\n" }, { "path": "fuse/doc.go", "content": "// Package fuse implements a FUSE layer for brig.\n// Using it, a repository may be represented as a \"normal\" directory.\n// There are three different structs in the FUSE API:\n//\n// - fuse.Node : A file or a directory (depending on it's type)\n//\t - fuse.FS : The filesystem. Used to find out the root node.\n// - fuse.Handle: An open file.\n//\n// This implementation offers File (a fuse.Node and fuse.Handle),\n// Dir (fuse.Node) and FS (fuse.FS).\n//\n// Fuse will call the respective handlers if it needs information about your\n// nodes. Each request handlers will usually get a `ctx` used to cancel\n// operations, a request structure `req` with detailed query infos and\n// a response structure `resp` where results are written. Usually the request\n// handlers might return an error or a new node/handle/fs.\n//\n// Every request handle that may run for a long time should be\n// made interruptible. Especially read and write operations should\n// check the ctx.Done() channel passed to each request handler.\npackage fuse\n" }, { "path": "fuse/file.go", "content": "// +build !windows\n\npackage fuse\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"os\"\n\t\"sync\"\n\n\tlog \"github.com/sirupsen/logrus\"\n\n\t\"bazil.org/fuse\"\n\t\"bazil.org/fuse/fs\"\n)\n\nvar (\n\t// ErrNotCached is returned in offline mode when we don't have a file\n\tErrNotCached = errors.New(\"content is not cached and need to be downloaded\")\n\n\t// ErrTooManyWriters is returned when writers counter is about to be overfilled\n\tErrTooManyWriters = errors.New(\"too many writers for the file\")\n)\n\n// File is a file inside a directory.\ntype File struct {\n\tmu sync.Mutex // used during handle creation\n\tpath string\n\tm *Mount\n\thd *Handle\n}\n\n// Attr is called to get the stat(2) attributes of a file.\nfunc (fi *File) Attr(ctx context.Context, attr *fuse.Attr) error {\n\tdefer logPanic(\"file: attr\")\n\tlog.Debugf(\"fuse-file-attr: %v\", fi.path)\n\n\tinfo, err := fi.m.fs.Stat(fi.path)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdebugLog(\"exec file attr: %v\", fi.path)\n\n\tvar filePerm os.FileMode = 0640\n\tattr.Mode = filePerm\n\tif fi.m.options.Offline {\n\t\tisCached, err := fi.m.fs.IsCached(fi.path)\n\t\tif err != nil || !isCached {\n\t\t\tif err != nil {\n\t\t\t\tlog.Errorf(\"IsCached failed for %s with error : %v\", fi.path, err)\n\t\t\t}\n\t\t\t// Uncached file will be shown as symlink\n\t\t\t// We cannot read them in Offline mode,\n\t\t\t// but we can delete such link and overwrite its content\n\t\t\tattr.Mode = os.ModeSymlink | filePerm\n\t\t}\n\t}\n\tattr.Size = info.Size\n\tattr.Mtime = info.ModTime\n\tattr.Inode = info.Inode\n\n\t// Act like the file is owned by the user of the brig process.\n\tattr.Uid = uint32(os.Getuid())\n\tattr.Gid = uint32(os.Getgid())\n\n\t// tools like `du` rely on this for size calculation\n\t// (assuming every fs block takes actual storage, but we only emulate this\n\t// here for compatibility; see man 2 stat for the why for \"512\")\n\tattr.BlockSize = 4096\n\tattr.Blocks = info.Size / 512\n\tif info.Size%uint64(512) > 0 {\n\t\tattr.Blocks++\n\t}\n\n\treturn nil\n}\n\n// Open is called to get an opened handle of a file, suitable for reading and writing.\nfunc (fi *File) Open(ctx context.Context, req *fuse.OpenRequest, resp *fuse.OpenResponse) (fs.Handle, error) {\n\tdefer logPanic(\"file: open\")\n\tdebugLog(\"fuse-open: %s\", fi.path)\n\tlog.Debugf(\"fuse-file-open: %v with request %v\", fi.path, req)\n\n\t// Check if the file is actually available locally.\n\tif fi.m.options.Offline {\n\t\tisCached, err := fi.m.fs.IsCached(fi.path)\n\t\tif err != nil {\n\t\t\treturn nil, errorize(\"file-is-cached\", err)\n\t\t}\n\n\t\tif !isCached {\n\t\t\treturn nil, errorize(\"file-not-cached\", ErrNotCached)\n\t\t}\n\t}\n\n\tfd, err := fi.m.fs.Open(fi.path)\n\tif err != nil {\n\t\treturn nil, errorize(\"file-open\", err)\n\t}\n\n\tfi.mu.Lock()\n\tif fi.hd == nil {\n\t\thd := Handle{fd: fd, m: fi.m, wasModified: false}\n\t\tfi.hd = &hd\n\t}\n\tfi.hd.fd = fd\n\tfi.mu.Unlock()\n\n\tresp.Flags |= fuse.OpenKeepCache\n\treturn fi.hd, nil\n}\n\n// Setattr is called once an attribute of a file changes.\n// Most importantly, size changes are reported here, e.g. after truncating a\n// file, the size change is noticed here before Open() is called.\nfunc (fi *File) Setattr(ctx context.Context, req *fuse.SetattrRequest, resp *fuse.SetattrResponse) error {\n\tdefer logPanic(\"file: setattr\")\n\tlog.Debugf(\"fuse-file-setattr: request %v\", req)\n\n\t// This is called when any attribute of the file changes,\n\t// most importantly the file size. For example it is called when truncating\n\t// the file to zero bytes with a size change of `0`.\n\tswitch {\n\tcase req.Valid&fuse.SetattrSize != 0:\n\t\tif err := fi.hd.truncate(req.Size); err != nil {\n\t\t\treturn errorize(\"file-setattr-size\", err)\n\t\t}\n\tcase req.Valid&fuse.SetattrMtime != 0:\n\t\tif err := fi.m.fs.Touch(fi.path); err != nil {\n\t\t\treturn errorize(\"file-setattr-mtime\", err)\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// Fsync is called when any open buffers need to be written to disk.\n// Currently, fsync is completely ignored.\nfunc (fi *File) Fsync(ctx context.Context, req *fuse.FsyncRequest) error {\n\tdefer logPanic(\"file: fsync\")\n\tlog.Debugf(\"fuse-file-fsync: %v\", fi.path)\n\n\treturn nil\n}\n\n// Getxattr is called to get a single xattr (extended attribute) of a file.\nfunc (fi *File) Getxattr(ctx context.Context, req *fuse.GetxattrRequest, resp *fuse.GetxattrResponse) error {\n\tdefer logPanic(\"file: getxattr\")\n\n\t// Do not worry about req.Size\n\t// fuse will cut it to allowed size and report to the caller that buffer need to be larger\n\txattrs, err := getXattr(fi.m.fs, req.Name, fi.path)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tresp.Xattr = xattrs\n\treturn nil\n}\n\n// Setxattr is called by the setxattr syscall.\nfunc (fi *File) Setxattr(ctx context.Context, req *fuse.SetxattrRequest) error {\n\tdefer logPanic(\"file: setxattr\")\n\n\treturn setXattr(fi.m.fs, req.Name, fi.path, req.Xattr)\n}\n\n// Listxattr is called to list all xattrs of this file.\nfunc (fi *File) Listxattr(ctx context.Context, req *fuse.ListxattrRequest, resp *fuse.ListxattrResponse) error {\n\tdefer logPanic(\"file: listxattr\")\n\n\t// Do not worry about req.Size\n\t// fuse will cut it to allowed size and report to the caller that buffer need to be larger\n\tresp.Xattr = listXattr()\n\treturn nil\n}\n\n// Readlink reads a symbolic link.\n// This call is triggered when OS tries to see where symlink points\nfunc (fi *File) Readlink(ctx context.Context, req *fuse.ReadlinkRequest) (string, error) {\n\tlog.Debugf(\"fuse-file-readlink: %v\", fi.path)\n\tinfo, err := fi.m.fs.Stat(fi.path)\n\tif err != nil {\n\t\treturn \"/brig/backend/ipfs/\", err\n\t}\n\treturn fmt.Sprintf(\"/brig/backend/ipfs/%s\", info.BackendHash), nil\n}\n\n// Compile time checks to see which interfaces we implement:\n// Please update this list when modifying code here.\nvar _ = fs.Node(&File{})\nvar _ = fs.NodeFsyncer(&File{})\nvar _ = fs.NodeGetxattrer(&File{})\nvar _ = fs.NodeListxattrer(&File{})\nvar _ = fs.NodeOpener(&File{})\nvar _ = fs.NodeSetattrer(&File{})\nvar _ = fs.NodeReadlinker(&File{})\nvar _ = fs.NodeSetxattrer(&File{})\n\n// Other interfaces are available, but currently not needed or make sense:\n// var _ = fs.NodeRenamer(&File{})\n// var _ = fs.NodeReadlinker(&File{})\n// var _ = fs.NodeRemover(&File{})\n// var _ = fs.NodeRemovexattrer(&File{})\n// var _ = fs.NodeRequestLookuper(&File{})\n// var _ = fs.NodeAccesser(&File{})\n// var _ = fs.NodeForgetter(&File{})\n// var _ = fs.NodeGetattrer(&File{})\n// var _ = fs.NodeLinker(&File{})\n// var _ = fs.NodeMkdirer(&File{})\n// var _ = fs.NodeMknoder(&File{})\n// var _ = fs.NodeStringLookuper(&File{})\n// var _ = fs.NodeSymlinker(&File{})\n" }, { "path": "fuse/fs.go", "content": "// +build !windows\n\npackage fuse\n\nimport (\n\t\"bazil.org/fuse/fs\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\nconst (\n\tenableDebugLogs = false\n)\n\nfunc debugLog(format string, args ...interface{}) {\n\tif enableDebugLogs {\n\t\tlog.Debugf(format, args...)\n\t}\n}\n\n// Filesystem is the entry point to the fuse filesystem\ntype Filesystem struct {\n\troot string\n\tm *Mount\n}\n\n// Root returns the topmost directory node.\n// This depends on what the user choose to select,\n// but usually it's \"/\".\nfunc (fs *Filesystem) Root() (fs.Node, error) {\n\treturn &Directory{path: fs.root, m: fs.m}, nil\n}\n" }, { "path": "fuse/fstab.go", "content": "// +build !windows\n\npackage fuse\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"sort\"\n\t\"strings\"\n\n\t\"github.com/sahib/brig/util\"\n\t\"github.com/sahib/config\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// FsTabAdd adds the mount at `path` with `name` and `opts` to `cfg`.\n// It does not yet do the mounting.\nfunc FsTabAdd(cfg *config.Config, name, path string, opts MountOptions) error {\n\tfor _, key := range cfg.Keys() {\n\t\tif strings.HasSuffix(key, \".path\") {\n\t\t\tif cfg.String(key) == path {\n\t\t\t\treturn fmt.Errorf(\"mount `%s` at path `%s` already exists\", name, path)\n\t\t\t}\n\t\t}\n\t}\n\n\tif cfg.String(name+\".path\") != \"\" {\n\t\treturn fmt.Errorf(\"mount `%s` already exists\", name)\n\t}\n\n\tif err := cfg.SetString(name+\".path\", path); err != nil {\n\t\treturn err\n\t}\n\n\tif err := cfg.SetBool(name+\".read_only\", opts.ReadOnly); err != nil {\n\t\treturn err\n\t}\n\n\tif err := cfg.SetBool(name+\".offline\", opts.Offline); err != nil {\n\t\treturn err\n\t}\n\n\tif opts.Root == \"\" {\n\t\topts.Root = \"/\"\n\t}\n\n\treturn cfg.SetString(name+\".root\", opts.Root)\n}\n\n// FsTabRemove removes a mount. It does not directly unmount it,\n// call FsTabApply for this.\nfunc FsTabRemove(cfg *config.Config, name string) error {\n\tif !cfg.IsValidKey(name) {\n\t\treturn fmt.Errorf(\"no such mount: %v\", name)\n\t}\n\n\treturn cfg.Reset(name)\n}\n\n// FsTabUnmountAll will unmount all currently mounted mounts.\nfunc FsTabUnmountAll(cfg *config.Config, mounts *MountTable) error {\n\tmounts.mu.Lock()\n\tdefer mounts.mu.Unlock()\n\n\terrors := util.Errors{}\n\tfor _, key := range cfg.Keys() {\n\t\tif strings.HasSuffix(key, \".path\") {\n\t\t\tmountPath := cfg.String(key)\n\t\t\tlog.Debugf(\"Unmount key %s %s\", key, mountPath)\n\t\t\tif mountPath == \"\" {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tif err := mounts.unmount(mountPath); err != nil {\n\t\t\t\terrors = append(errors, err)\n\t\t\t}\n\t\t}\n\t}\n\n\treturn errors.ToErr()\n}\n\n// FsTabApply takes all configured mounts and makes sure that all of them are opened.\nfunc FsTabApply(cfg *config.Config, mounts *MountTable) error {\n\tmounts.mu.Lock()\n\tdefer mounts.mu.Unlock()\n\n\tmountPaths := make(map[string]*MountOptions)\n\tfor _, key := range cfg.Keys() {\n\t\tif strings.HasSuffix(key, \".path\") {\n\t\t\tmountPath := cfg.String(key)\n\n\t\t\tentry := &MountOptions{}\n\t\t\tmountPaths[mountPath] = entry\n\n\t\t\treadOnlyKey := key[:len(key)-len(\".path\")] + \".read_only\"\n\t\t\tentry.ReadOnly = cfg.Bool(readOnlyKey)\n\n\t\t\tofflineKey := key[:len(key)-len(\".path\")] + \".offline\"\n\t\t\tentry.Offline = cfg.Bool(offlineKey)\n\n\t\t\trootPathKey := key[:len(key)-len(\".path\")] + \".root\"\n\t\t\tentry.Root = cfg.String(rootPathKey)\n\t\t\tif entry.Root == \"\" {\n\t\t\t\tentry.Root = \"/\"\n\t\t\t}\n\t\t}\n\t}\n\n\terrors := util.Errors{}\n\tfor path, mount := range mounts.m {\n\t\t// Do not do anything when the path / options did not change.\n\t\topts, isConfigured := mountPaths[path]\n\t\tif isConfigured && mount.EqualOptions(*opts) {\n\t\t\tdelete(mountPaths, path)\n\t\t\tcontinue\n\t\t}\n\n\t\tif err := mounts.unmount(path); err != nil {\n\t\t\terrors = append(errors, err)\n\t\t}\n\t}\n\n\tfor mountPath, options := range mountPaths {\n\t\tif err := os.MkdirAll(mountPath, 0700); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tif _, err := mounts.addMount(mountPath, *options); err != nil {\n\t\t\terrors = append(errors, err)\n\t\t}\n\t}\n\n\treturn errors.ToErr()\n}\n\n// FsTabEntry is a representation of one entry in the filesystem tab.\ntype FsTabEntry struct {\n\tName string\n\tPath string\n\tRoot string\n\tActive bool\n\tReadOnly bool\n\tOffline bool\n}\n\n// FsTabList lists all entries in the filesystem tab in a nice way.\nfunc FsTabList(cfg *config.Config, mounts *MountTable) ([]FsTabEntry, error) {\n\tmounts.mu.Lock()\n\tdefer mounts.mu.Unlock()\n\n\tmountMap := make(map[string]*FsTabEntry)\n\tfor _, key := range cfg.Keys() {\n\t\tsplit := strings.Split(key, \".\")\n\t\tif len(split) < 3 || split[0] != \"mounts\" {\n\t\t\tcontinue\n\t\t}\n\n\t\tmountName := split[1]\n\t\tif _, ok := mountMap[mountName]; !ok {\n\t\t\tmountMap[mountName] = &FsTabEntry{}\n\t\t}\n\n\t\tswitch split[2] {\n\t\tcase \"path\":\n\t\t\tpath := cfg.String(key)\n\t\t\tmountMap[mountName].Path = path\n\n\t\t\t_, isActive := mounts.m[path]\n\t\t\tmountMap[mountName].Active = isActive\n\t\tcase \"read_only\":\n\t\t\tmountMap[mountName].ReadOnly = cfg.Bool(key)\n\t\tcase \"offline\":\n\t\t\tmountMap[mountName].Offline = cfg.Bool(key)\n\t\tcase \"root\":\n\t\t\tmountMap[mountName].Root = cfg.String(key)\n\t\t}\n\t}\n\n\tsortedMounts := []FsTabEntry{}\n\tfor name, entry := range mountMap {\n\t\tentry.Name = name\n\t\tsortedMounts = append(sortedMounts, *entry)\n\t}\n\n\tsort.Slice(sortedMounts, func(i, j int) bool {\n\t\treturn sortedMounts[i].Name < sortedMounts[j].Name\n\t})\n\n\treturn sortedMounts, nil\n}\n" }, { "path": "fuse/fuse_test.go", "content": "// +build !windows\n\npackage fuse\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/binary\"\n\t\"flag\"\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"net/http\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"syscall\"\n\t\"testing\"\n\n\t\"github.com/sahib/brig/catfs\"\n\t\"github.com/sahib/brig/catfs/mio/pagecache/mdcache\"\n\t\"github.com/sahib/brig/defaults\"\n\t\"github.com/sahib/brig/util/testutil\"\n\t\"github.com/sahib/config\"\n\tlog \"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"bazil.org/fuse/fs/fstestutil/spawntest\"\n\t\"bazil.org/fuse/fs/fstestutil/spawntest/httpjson\"\n)\n\n// The routines which manage fuse layer\n// and OS dependent code (os.Open, and similar) MUST BE SEPARATE OS PROCESSES.\n// Note: not different go routines but processes!\n// See https://github.com/bazil/fuse/issues/264#issuecomment-727269770\n// This separation happens automatically during normal brig operations, but\n// TESTING FUSE LAYER IN GO IS VERY TRICKY.\n// See brig relevant discussion at\n// https://github.com/sahib/brig/pull/77#issuecomment-754831080\n// However this issue is general for any go program from version 1.9,\n// as can be seen in references to the issue.\n//\n// bazil/fuse offers \"bazil.org/fuse/fs/fstestutil/spawntest\"\n// infrastructure which helps run tests in different communicating via socket processes.\n\nfunc init() {\n\tlog.SetLevel(log.ErrorLevel)\n}\n\nfunc TestMain(m *testing.M) {\n\thelpers.AddFlag(flag.CommandLine)\n\tflag.Parse()\n\thelpers.RunIfNeeded()\n\tos.Exit(m.Run())\n}\n\ntype fuseCatFSHelp struct{}\n\n// These helpers will be requested from test and executed on the server\n// which is managing catfs-fuse connection (started within test)\nfunc (fch *fuseCatFSHelp) ServeHTTP(w http.ResponseWriter, req *http.Request) {\n\tswitch req.URL.Path {\n\tcase \"/mount\":\n\t\thttpjson.ServePOST(fch.makeCatfsAndFuseMount).ServeHTTP(w, req)\n\tcase \"/unmount\":\n\t\thttpjson.ServePOST(fch.unmountFuseAndCloseDummyCatFS).ServeHTTP(w, req)\n\tcase \"/fuseReMount\":\n\t\thttpjson.ServePOST(fch.makeFuseReMount).ServeHTTP(w, req)\n\tcase \"/catfsStage\":\n\t\thttpjson.ServePOST(fch.catfsStage).ServeHTTP(w, req)\n\tcase \"/catfsGetData\":\n\t\thttpjson.ServePOST(fch.catfsGetData).ServeHTTP(w, req)\n\tdefault:\n\t\thttp.NotFound(w, req)\n\t}\n}\n\nfunc makeDummyCatFS(dbPath string) (catfsFuseInfo, error) {\n\tbackend := catfs.NewMemFsBackend()\n\towner := \"alice\"\n\n\tcfg, err := config.Open(nil, defaults.Defaults, config.StrictnessPanic)\n\tif err != nil {\n\t\tlog.Fatalf(\"Could not get default catFS config: %v\", err)\n\t\treturn catfsFuseInfo{}, err\n\t}\n\n\tmdc, err := mdcache.New(mdcache.Options{\n\t\tMaxMemoryUsage: 1024 * 1024 * 1024,\n\t})\n\n\tif err != nil {\n\t\tlog.Fatalf(\"unable to instance : %v\", err)\n\t\treturn catfsFuseInfo{}, err\n\t}\n\n\tcfs, err := catfs.NewFilesystem(\n\t\tbackend,\n\t\tdbPath,\n\t\towner,\n\t\tfalse,\n\t\tcfg.Section(\"fs\"),\n\t\tnil,\n\t\tmdc,\n\t)\n\n\tif err != nil {\n\t\tlog.Fatalf(\"Failed to create catfs filesystem: %v\", err)\n\t\treturn catfsFuseInfo{}, err\n\t}\n\tcfInfo := catfsFuseInfo{}\n\tcfInfo.cfs = cfs\n\tcfInfo.dbPath = dbPath\n\n\treturn cfInfo, err\n}\n\ntype nothing struct{} // use it to send empty request or responses to server\n\ntype catfsFuseInfo struct {\n\tcfs *catfs.FS\n\tdbPath string\n\tfuseMount *Mount\n}\n\n// cfInfo will be in the global space for the server\n// which manage fuse mount connection to the catFS\nvar cfInfo catfsFuseInfo\n\ntype mountingRequest struct {\n\tDbPath string\n\tMntPath string\n\tOpts MountOptions\n}\n\nfunc (fch *fuseCatFSHelp) makeCatfsAndFuseMount(ctx context.Context, req mountingRequest) (*nothing, error) {\n\tvar err error\n\tcfInfo, err = makeDummyCatFS(req.DbPath)\n\tif err != nil {\n\t\tlog.Errorf(\"cannot make catFS in %v\", cfInfo.dbPath)\n\t\treturn ¬hing{}, err\n\t}\n\n\tfuseMount, err := makeFuseMount(cfInfo.cfs, req.MntPath, req.Opts)\n\tif err != nil {\n\t\tlog.Errorf(\"cannot mount catfs fuse file system to %v\", req.MntPath)\n\t\treturn ¬hing{}, err\n\t}\n\tcfInfo.fuseMount = fuseMount\n\treturn ¬hing{}, err\n}\n\nfunc (fch *fuseCatFSHelp) makeFuseReMount(ctx context.Context, req mountingRequest) (*nothing, error) {\n\tfuseMount, err := makeFuseMount(cfInfo.cfs, req.MntPath, req.Opts)\n\tif err != nil {\n\t\tlog.Errorf(\"cannot mount catfs fuse file system to %v\", req.MntPath)\n\t\treturn ¬hing{}, err\n\t}\n\tcfInfo.fuseMount = fuseMount\n\treturn ¬hing{}, err\n}\n\nfunc (fch *fuseCatFSHelp) unmountFuseAndCloseDummyCatFS(ctx context.Context, req nothing) (*nothing, error) {\n\tdefer os.RemoveAll(cfInfo.fuseMount.Dir)\n\tdefer os.RemoveAll(cfInfo.dbPath)\n\t// first unmount fuse directory\n\tif err := lazyUnmount(cfInfo.fuseMount.Dir); err != nil {\n\t\tskipableErr := \"exit status 1: fusermount: entry for \" + cfInfo.fuseMount.Dir + \" not found in /etc/mtab\"\n\t\tlog.Debug(skipableErr)\n\t\tif err.Error() != skipableErr {\n\t\t\treturn ¬hing{}, err\n\t\t}\n\t}\n\n\t// now close catFS\n\terr := cfInfo.cfs.Close()\n\tif err != nil {\n\t\tlog.Fatalf(\"Could not close catfs filesystem: %v\", err)\n\t}\n\treturn ¬hing{}, err\n}\n\nfunc makeFuseMount(cfs *catfs.FS, mntPath string, opts MountOptions) (*Mount, error) {\n\t// Make sure to unmount any mounts that are there.\n\t// Possibly there are some leftovers from previous failed runs.\n\tif err := lazyUnmount(mntPath); err != nil {\n\t\tskipableErr := \"exit status 1: fusermount: entry for \" + mntPath + \" not found in /etc/mtab\"\n\t\tlog.Debug(skipableErr)\n\t\tif err.Error() != skipableErr {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tif err := os.MkdirAll(mntPath, 0777); err != nil {\n\t\tlog.Fatalf(\"Unable to create empty mount dir: %v\", err)\n\t\treturn nil, err\n\t}\n\n\tmount, err := NewMount(cfs, mntPath, nil, opts)\n\tif err != nil {\n\t\tlog.Fatalf(\"Cannot create mount: %v\", err)\n\t\treturn nil, err\n\t}\n\treturn mount, err\n}\n\ntype catfsPayload struct {\n\tPath string\n\tData []byte\n}\n\nfunc (fch *fuseCatFSHelp) catfsStage(ctx context.Context, req catfsPayload) (*nothing, error) {\n\terr := cfInfo.cfs.Stage(req.Path, bytes.NewReader(req.Data))\n\treturn ¬hing{}, err\n}\n\n// Get data from a file stored by catFS\nfunc (fch *fuseCatFSHelp) catfsGetData(ctx context.Context, req catfsPayload) (*catfsPayload, error) {\n\tout := catfsPayload{}\n\tout.Path = req.Path\n\n\tstream, err := cfInfo.cfs.Cat(req.Path)\n\tif err != nil {\n\t\tlog.Fatalf(\"Could not get stream for a catfs file: %v\", err)\n\t\treturn &out, err\n\t}\n\tresult := bytes.NewBuffer(nil)\n\t_, err = stream.WriteTo(result)\n\tif err != nil {\n\t\tlog.Fatalf(\"Streaming to a buffer failed: %v\", err)\n\t\treturn &out, err\n\t}\n\tout.Data = result.Bytes()\n\n\treturn &out, err\n}\n\nvar helpers spawntest.Registry\nvar fuseCatFSHelper = helpers.Register(\"fuseCatFSHelp\", &fuseCatFSHelp{})\n\ntype mountInfo struct { // fuse related info available to OS layer\n\tDir string\n\tOpts MountOptions\n}\n\n// Call helper for unmount and cleanup\nfunc callUnMount(ctx context.Context, t testing.TB, control *spawntest.Control) {\n\tif err := control.JSON(\"/unmount\").Call(ctx, nothing{}, ¬hing{}); err != nil {\n\t\tt.Fatalf(\"calling helper: %v\", err)\n\t}\n}\n\n// Spawns helper, prepare catFS, connects it to fuse layer, and execute function f\nfunc withMount(t testing.TB, opts MountOptions, f func(ctx context.Context, control *spawntest.Control, mount *mountInfo)) {\n\t// set up mounts\n\tctx, cancel := context.WithCancel(context.Background())\n\tdefer cancel()\n\n\tcontrol := fuseCatFSHelper.Spawn(ctx, t)\n\tdefer control.Close()\n\n\tdbPath, err := ioutil.TempDir(\"\", \"catfs-db-test\")\n\tif err != nil {\n\t\tt.Fatalf(\"Failed to create temp dir for catFS: %v\", err)\n\t}\n\n\treq := mountingRequest{\n\t\tDbPath: dbPath,\n\t\tMntPath: filepath.Join(os.TempDir(), \"catfs-fuse-mountdir\"),\n\t\tOpts: opts,\n\t}\n\n\tif err := control.JSON(\"/mount\").Call(ctx, req, ¬hing{}); err != nil {\n\t\tt.Fatalf(\"calling helper: %v\", err)\n\t}\n\tdefer callUnMount(ctx, t, control)\n\n\t// function which required mounts\n\tf(ctx, control, &mountInfo{\n\t\tDir: req.MntPath,\n\t\tOpts: req.Opts,\n\t})\n}\n\nfunc checkFuseFileMatchToCatFS(ctx context.Context, t *testing.T, control *spawntest.Control, fusePath string, catfsPath string) {\n\t// checks if OS file content matches catFS file content\n\tfuseData, err := ioutil.ReadFile(fusePath)\n\trequire.NoError(t, err)\n\n\t// is catFS seeing the same data\n\tcheckCatfsFileContent(ctx, t, control, catfsPath, fuseData)\n}\n\nfunc checkCatfsFileContent(ctx context.Context, t *testing.T, control *spawntest.Control, catfsPath string, expected []byte) {\n\treq := catfsPayload{Path: catfsPath}\n\tout := catfsPayload{}\n\trequire.NoError(t, control.JSON(\"/catfsGetData\").Call(ctx, req, &out))\n\trequire.Equal(t, len(out.Data), len(expected))\n\tif out.Data == nil {\n\t\t// this is special for the 0 length data\n\t\tout.Data = []byte{}\n\t}\n\trequire.Equal(t, out.Data, expected)\n}\n\n// Finally we ready to do tests\n\n// Tests for spawntest infrastructure related tests\n\n// Just checks that our catfsStage interface to catFS does not error out\nfunc TestCatfsStage(t *testing.T) {\n\twithMount(t, MountOptions{}, func(ctx context.Context, control *spawntest.Control, mount *mountInfo) {\n\t\tdataIn := []byte{1, 2, 3, 4}\n\t\tfilePath := \"StagingTest.bin\"\n\t\treq := catfsPayload{Path: filePath, Data: dataIn}\n\t\trequire.NoError(t, control.JSON(\"/catfsStage\").Call(ctx, req, ¬hing{}))\n\t})\n}\n\nfunc TestCatfsGetData(t *testing.T) {\n\twithMount(t, MountOptions{}, func(ctx context.Context, control *spawntest.Control, mount *mountInfo) {\n\t\tdataIn := []byte{1, 2, 3, 4}\n\t\tfilePath := \"StageAndReadTest.bin\"\n\t\treq := catfsPayload{Path: filePath, Data: dataIn}\n\t\trequire.NoError(t, control.JSON(\"/catfsStage\").Call(ctx, req, ¬hing{}))\n\n\t\treq.Data = []byte{}\n\t\tout := catfsPayload{}\n\t\trequire.NoError(t, control.JSON(\"/catfsGetData\").Call(ctx, req, &out))\n\t\trequire.Equal(t, out.Data, dataIn)\n\t})\n}\n\n// Main fuse layer tests\n\nvar (\n\tDataSizes = []int64{\n\t\t0, 1, 2, 4, 8, 16, 32, 64, 1024,\n\t\t2048, 4095, 4096, 4097, 147611,\n\t\t2*1024*1024 + 123, // in case if we have buffer size interference\n\t}\n)\n\nfunc TestRead(t *testing.T) {\n\twithMount(t, MountOptions{}, func(ctx context.Context, control *spawntest.Control, mount *mountInfo) {\n\t\tfor _, size := range DataSizes {\n\t\t\tt.Run(fmt.Sprintf(\"%d\", size), func(t *testing.T) {\n\t\t\t\thelloData := testutil.CreateDummyBuf(size)\n\n\t\t\t\t// Add a simple file:\n\t\t\t\tcatfsFilePath := fmt.Sprintf(\"/hello_from_catfs_%d\", size)\n\t\t\t\treq := catfsPayload{Path: catfsFilePath, Data: helloData}\n\t\t\t\trequire.NoError(t, control.JSON(\"/catfsStage\").Call(ctx, req, ¬hing{}))\n\t\t\t\tcheckCatfsFileContent(ctx, t, control, catfsFilePath, helloData)\n\n\t\t\t\tfuseFilePath := filepath.Join(mount.Dir, catfsFilePath)\n\t\t\t\tcheckFuseFileMatchToCatFS(ctx, t, control, fuseFilePath, catfsFilePath)\n\t\t\t})\n\t\t}\n\t})\n}\n\nfunc TestFileXattr(t *testing.T) {\n\twithMount(t, MountOptions{}, func(ctx context.Context, control *spawntest.Control, mount *mountInfo) {\n\t\tsize := int64(4)\n\t\thelloData := testutil.CreateDummyBuf(size)\n\n\t\t// Add a simple file:\n\t\tcatfsFilePath := fmt.Sprintf(\"/hello_from_catfs_%d\", size)\n\t\treq := catfsPayload{Path: catfsFilePath, Data: helloData}\n\t\trequire.NoError(t, control.JSON(\"/catfsStage\").Call(ctx, req, ¬hing{}))\n\t\tcheckCatfsFileContent(ctx, t, control, catfsFilePath, helloData)\n\n\t\tfuseFilePath := filepath.Join(mount.Dir, catfsFilePath)\n\n\t\t// no let's see all the extended attributes list\n\t\tresponse := make([]byte, 1024*4) // large buffer to fit everything\n\t\tsz, err := syscall.Listxattr(fuseFilePath, response)\n\t\trequire.NoError(t, err)\n\t\tresponse = response[:sz]\n\t\treceivedAttrs := bytes.Split(response, []byte{0})\n\t\t// every response should belong to valid attributes\n\t\tfor _, attr := range receivedAttrs {\n\t\t\tif len(attr) == 0 {\n\t\t\t\t// protecting against empty chunk after split delimiter\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\t_, ok := xattrMap[string(attr)]\n\t\t\trequire.Truef(t, ok, \"Invalid extended attribute '%s'\", attr)\n\t\t}\n\t\t// every valid attribute should be in received Attrs list\n\t\tfor attr := range xattrMap {\n\t\t\trequire.Containsf(t, receivedAttrs, []uint8(attr), \"Received attributes are missing '%s'\", attr)\n\t\t}\n\t\t// now let's check some attributes values\n\t\t// Note hashes are hard without direct access to catfs\n\t\t// which is accessed in different process\n\t\tresponse = make([]byte, 64) // large buffer to fit everything\n\t\tsz, err = syscall.Getxattr(fuseFilePath, \"user.brig.pinned\", response)\n\t\trequire.NoError(t, err)\n\t\tresponse = response[:sz]\n\t\trequire.Equal(t, \"yes\", string(response))\n\n\t\tresponse = make([]byte, 64) // large buffer to fit everything\n\t\tsz, err = syscall.Getxattr(fuseFilePath, \"user.brig.explicitly_pinned\", response)\n\t\trequire.NoError(t, err)\n\t\tresponse = response[:sz]\n\t\trequire.Equal(t, \"no\", string(response))\n\t})\n}\n\nfunc TestWrite(t *testing.T) {\n\twithMount(t, MountOptions{}, func(ctx context.Context, control *spawntest.Control, mount *mountInfo) {\n\t\tfor _, size := range DataSizes {\n\t\t\tt.Run(fmt.Sprintf(\"%d\", size), func(t *testing.T) {\n\t\t\t\thelloData := testutil.CreateDummyBuf(size)\n\n\t\t\t\tcatfsFilePath := fmt.Sprintf(\"/hello_from_fuse%d\", size)\n\t\t\t\tfuseFilePath := filepath.Join(mount.Dir, catfsFilePath)\n\n\t\t\t\t// Write a simple file via the fuse layer:\n\t\t\t\terr := ioutil.WriteFile(fuseFilePath, helloData, 0644)\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Fatalf(\"Could not write simple file via fuse layer: %v\", err)\n\t\t\t\t}\n\t\t\t\tcheckCatfsFileContent(ctx, t, control, catfsFilePath, helloData)\n\t\t\t})\n\t\t}\n\t})\n}\n\n// Regression test for copying larger file to the mount.\nfunc TestTouchWrite(t *testing.T) {\n\twithMount(t, MountOptions{}, func(ctx context.Context, control *spawntest.Control, mount *mountInfo) {\n\t\tfor _, size := range DataSizes {\n\t\t\tt.Run(fmt.Sprintf(\"%d\", size), func(t *testing.T) {\n\n\t\t\t\tcatfsFilePath := fmt.Sprintf(\"/empty_at_creation_by_catfs_%d\", size)\n\t\t\t\treq := catfsPayload{Path: catfsFilePath, Data: []byte{}}\n\n\t\t\t\trequire.NoError(t, control.JSON(\"/catfsStage\").Call(ctx, req, ¬hing{}))\n\t\t\t\tcheckCatfsFileContent(ctx, t, control, catfsFilePath, req.Data)\n\n\t\t\t\tfuseFilePath := filepath.Join(mount.Dir, catfsFilePath)\n\n\t\t\t\t// Write a simple file via the fuse layer:\n\t\t\t\thelloData := testutil.CreateDummyBuf(size)\n\t\t\t\terr := ioutil.WriteFile(fuseFilePath, helloData, 0644)\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Fatalf(\"Could not write simple file via fuse layer: %v\", err)\n\t\t\t\t}\n\t\t\t\tcheckCatfsFileContent(ctx, t, control, catfsFilePath, helloData)\n\t\t\t})\n\t\t}\n\t})\n}\n\n// Regression test for copying a file to a subdirectory.\nfunc TestTouchWriteSubdir(t *testing.T) {\n\twithMount(t, MountOptions{}, func(ctx context.Context, control *spawntest.Control, mount *mountInfo) {\n\t\tfile := \"donald.png\"\n\t\tsubDirPath := \"sub\"\n\t\tcatfsFilePath := filepath.Join(subDirPath, file)\n\n\t\tfuseSubDirPath := filepath.Join(mount.Dir, subDirPath)\n\t\tfuseFilePath := filepath.Join(fuseSubDirPath, file)\n\n\t\trequire.NoError(t, os.Mkdir(fuseSubDirPath, 0644))\n\n\t\texpected := []byte{1, 2, 3}\n\t\trequire.NoError(t, ioutil.WriteFile(fuseFilePath, expected, 0644))\n\n\t\tcheckCatfsFileContent(ctx, t, control, catfsFilePath, expected)\n\t})\n}\n\nfunc TestReadOnlyFs(t *testing.T) {\n\topts := MountOptions{\n\t\tReadOnly: true,\n\t}\n\twithMount(t, opts, func(ctx context.Context, control *spawntest.Control, mount *mountInfo) {\n\t\txData := []byte{1, 2, 3}\n\t\treq := catfsPayload{Path: \"/x.png\", Data: xData}\n\n\t\trequire.NoError(t, control.JSON(\"/catfsStage\").Call(ctx, req, ¬hing{}))\n\t\tcheckCatfsFileContent(ctx, t, control, \"x.png\", xData)\n\n\t\t// Do some allowed io to check if the fs is actually working.\n\t\t// The test does not check on the kind of errors otherwise.\n\t\txPath := filepath.Join(mount.Dir, \"x.png\")\n\t\tcheckFuseFileMatchToCatFS(ctx, t, control, xPath, \"x.png\")\n\n\t\t// Try creating a new file:\n\t\tyPath := filepath.Join(mount.Dir, \"y.png\")\n\t\trequire.NotNil(t, ioutil.WriteFile(yPath, []byte{4, 5, 6}, 0600))\n\n\t\t// Try modifying an existing file:\n\t\trequire.NotNil(t, ioutil.WriteFile(xPath, []byte{4, 5, 6}, 0600))\n\n\t\tdirPath := filepath.Join(mount.Dir, \"sub\")\n\t\trequire.NotNil(t, os.Mkdir(dirPath, 0644))\n\t})\n}\n\nfunc TestWithRoot(t *testing.T) {\n\twithMount(t, MountOptions{}, func(ctx context.Context, control *spawntest.Control, mount *mountInfo) {\n\t\tdata := []byte{1, 2, 3}\n\t\t// Populate catFS with some files in different directories\n\t\treq := catfsPayload{Path: \"/u.png\", Data: data}\n\n\t\trequire.NoError(t, control.JSON(\"/catfsStage\").Call(ctx, req, ¬hing{}))\n\t\tcheckCatfsFileContent(ctx, t, control, req.Path, data)\n\t\tcheckFuseFileMatchToCatFS(ctx, t, control, filepath.Join(mount.Dir, req.Path), req.Path)\n\n\t\tdata = []byte{2, 3, 4}\n\t\treq = catfsPayload{Path: \"/a/x.png\", Data: data}\n\t\trequire.NoError(t, control.JSON(\"/catfsStage\").Call(ctx, req, ¬hing{}))\n\t\tcheckCatfsFileContent(ctx, t, control, req.Path, data)\n\t\tcheckFuseFileMatchToCatFS(ctx, t, control, filepath.Join(mount.Dir, req.Path), req.Path)\n\n\t\tdata = []byte{3, 4, 5}\n\t\treq = catfsPayload{Path: \"/a/b/y.png\", Data: data}\n\t\trequire.NoError(t, control.JSON(\"/catfsStage\").Call(ctx, req, ¬hing{}))\n\t\tcheckCatfsFileContent(ctx, t, control, req.Path, data)\n\t\tcheckFuseFileMatchToCatFS(ctx, t, control, filepath.Join(mount.Dir, req.Path), req.Path)\n\n\t\tdata = []byte{4, 5, 6}\n\t\treq = catfsPayload{Path: \"/a/b/c/z.png\", Data: data}\n\t\trequire.NoError(t, control.JSON(\"/catfsStage\").Call(ctx, req, ¬hing{}))\n\t\tcheckCatfsFileContent(ctx, t, control, req.Path, data)\n\t\tcheckFuseFileMatchToCatFS(ctx, t, control, filepath.Join(mount.Dir, req.Path), req.Path)\n\n\t\t// Now we need to remount fuse with different root directory\n\t\tremntReq := mountingRequest{\n\t\t\tMntPath: mount.Dir,\n\t\t\tOpts: MountOptions{Root: \"/a/b\"},\n\t\t}\n\t\trequire.NoError(t, control.JSON(\"/fuseReMount\").Call(ctx, remntReq, ¬hing{}))\n\t\tmount.Opts = remntReq.Opts // update with new mount options\n\n\t\t// See if fuse indeed provides different root\n\t\t// Read already existing file\n\t\tyPath := filepath.Join(mount.Dir, \"y.png\")\n\t\tcheckFuseFileMatchToCatFS(ctx, t, control, yPath, \"/a/b/y.png\")\n\n\t\t// Write to a new file\n\t\tdata = []byte{5, 6, 7}\n\t\tnewPath := filepath.Join(mount.Dir, \"new.png\")\n\n\t\trequire.NoError(t, ioutil.WriteFile(newPath, data, 0644))\n\t\tcheckCatfsFileContent(ctx, t, control, \"/a/b/new.png\", data)\n\n\t\t// Attempt to read file above mounted root\n\t\tinAccessiblePath := filepath.Join(mount.Dir, \"u.png\")\n\t\t_, err := ioutil.ReadFile(inAccessiblePath)\n\t\trequire.NotNil(t, err)\n\t})\n}\n\n// Benchmarks\n\nvar (\n\tBenchmarkDataSizes = []int64{\n\t\t0,\n\t\t1024, 2 * 1024, 16 * 1024, 64 * 1024, 128 * 1024,\n\t\t1 * 1024 * 1024, 16 * 1024 * 1024,\n\t}\n)\n\nfunc stageAndRead(ctx context.Context, b *testing.B, control *spawntest.Control, mount *mountInfo, label string, data []byte) {\n\tsize := len(data)\n\t// stage data to catFS\n\tcatfsFilePath := fmt.Sprintf(\"%s_file_%d\", label, size)\n\treq := catfsPayload{Path: catfsFilePath, Data: data}\n\trequire.NoError(b, control.JSON(\"/catfsStage\").Call(ctx, req, ¬hing{}))\n\tfuseFilePath := filepath.Join(mount.Dir, catfsFilePath)\n\n\t// Read it back via fuse\n\tb.Run(fmt.Sprintf(\"%s_Size_%d\", label, size), func(b *testing.B) {\n\t\tfor n := 0; n < b.N; n++ {\n\t\t\tioutil.ReadFile(fuseFilePath)\n\t\t}\n\t})\n}\n\nfunc BenchmarkRead(b *testing.B) {\n\twithMount(b, MountOptions{}, func(ctx context.Context, control *spawntest.Control, mount *mountInfo) {\n\t\tfor _, size := range BenchmarkDataSizes {\n\t\t\t// Check how fast is readout of a file with compressible content\n\t\t\tdata := testutil.CreateDummyBuf(size)\n\t\t\tstageAndRead(ctx, b, control, mount, \"CompressibleContent\", data)\n\n\t\t\t// Check how fast is readout of a file with random/uncompressible content\n\t\t\tdata = testutil.CreateRandomDummyBuf(size, 1)\n\t\t\tstageAndRead(ctx, b, control, mount, \"RandomContent\", data)\n\t\t}\n\t})\n}\n\nfunc writeDataNtimes(b *testing.B, data []byte, ntimes int) {\n\t// Writing could be very space demanding even for a small size,\n\t// Since benchmark runs many-many times, it will consume a lot of space.\n\t// We have to remount everything every time to start with clean catFS DB.\n\t// Consequently, this test takes long time, since mounting is long operation.\n\trequire.True(b, ntimes > 0, \"ntimes must be positive\")\n\t// note ntimes =0 is bad too,\n\t// since execution time between StartTimer/StopTimer is too short/jittery\n\t// and benchmarks run forever\n\tlabel := \"dummy\"\n\tsize := len(data)\n\tfor n := 0; n < b.N; n++ {\n\t\tb.StopTimer()\n\t\twithMount(b, MountOptions{}, func(ctx context.Context, control *spawntest.Control, mount *mountInfo) {\n\t\t\t// Check how fast is write to a file with compressible content\n\t\t\tcatfsFilePath := fmt.Sprintf(\"%s_file_%d\", label, size)\n\t\t\tfuseFilePath := filepath.Join(mount.Dir, catfsFilePath)\n\n\t\t\tb.StartTimer()\n\t\t\tfor i := 0; i < ntimes; i++ {\n\t\t\t\tif len(data) > 0 {\n\t\t\t\t\t// modification of one byte is enough\n\t\t\t\t\t// to generate new encrypted content for the backend\n\t\t\t\t\tbinary.LittleEndian.PutUint64(data[0:8], uint64(i))\n\t\t\t\t}\n\t\t\t\trequire.NoError(b, ioutil.WriteFile(fuseFilePath, data, 0644))\n\t\t\t}\n\t\t\tb.StopTimer()\n\t\t})\n\t}\n}\n\nvar (\n\t// keep this low or you might run out of space\n\tNumberOfOverWrites = []int{\n\t\t1, 2, 5,\n\t}\n)\n\nfunc BenchmarkWrite(b *testing.B) {\n\tsize := int64(10 * 1024 * 1024)\n\n\tfor _, Ntimes := range NumberOfOverWrites {\n\t\t// Check how fast is write to a file with compressible content\n\t\tdata := testutil.CreateDummyBuf(size)\n\t\tprefix := fmt.Sprintf(\"Owerwrite_%d\", Ntimes)\n\t\tlabel := fmt.Sprintf(\"%s/CompressibleContent_Size_%d\", prefix, size)\n\t\tb.Run(label, func(b *testing.B) {\n\t\t\twriteDataNtimes(b, data, Ntimes)\n\t\t})\n\n\t\t// Check how fast is write to a file with random/uncompressible content\n\t\tdata = testutil.CreateRandomDummyBuf(size, 1)\n\t\tlabel = fmt.Sprintf(\"%s/RandomContent_Size_%d\", prefix, size)\n\t\tb.Run(label, func(b *testing.B) {\n\t\t\twriteDataNtimes(b, data, Ntimes)\n\t\t})\n\t}\n}\n" }, { "path": "fuse/fusetest/client.go", "content": "package fusetest\n\nimport (\n\t\"context\"\n\t\"net\"\n\t\"net/http\"\n\n\t\"github.com/sahib/brig/util\"\n)\n\n// Client allows controlling the\ntype Client struct {\n\thttpClient *http.Client\n}\n\n// Dial returns a client suitable for controlling a fusetest server.\nfunc Dial(url string) (*Client, error) {\n\tscheme, addr, err := util.URLToSchemeAndAddr(url)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\thttpClient := &http.Client{\n\t\tTransport: &http.Transport{\n\t\t\tDialContext: func(_ context.Context, _, _ string) (net.Conn, error) {\n\t\t\t\treturn net.Dial(scheme, addr)\n\t\t\t},\n\t\t},\n\t}\n\treturn &Client{\n\t\thttpClient: httpClient,\n\t}, nil\n}\n\n// QuitServer sends a command that tells the server to quit.\n// The request will block until the quit was carried out.\nfunc (ctl *Client) QuitServer() error {\n\treq, err := http.NewRequest(\"GET\", \"/quit\", nil)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t_, err = ctl.httpClient.Do(req)\n\treturn err\n}\n" }, { "path": "fuse/fusetest/doc.go", "content": "// Package fusetest offers an easy way to test our fuse code.\n//\n// What this does is start another process with an HTTP server in it.\n// Beside the HTTP server the fuse mount is mounted at a specified path,\n// with the specified options. A client can connect to the server and control\n// it and/or a client program can access files in the fuse mount.\n//\n// The reason why this is another process is an issue with Go:\n// When serving and accessing the FUSE mount in the same we might enter\n// an unrecoverable deadlock where file I/O related syscalls get stuck\n// because the go routine that serve this syscall live in the same process\n// but do not get called because only N parallel go routines can be run.\n//\n// Reference:\n//\n// * https://github.com/bazil/fuse/issues/264#issuecomment-727269770\n// * https://github.com/sahib/brig/pull/77#issuecomment-754831080\n//\n// bazil/fuse offers an spawntest utility that does something very similar,\n// but we also this package in benchmarks and spawntest expects to be called\n// from tests and thus requires the testing package.\npackage fusetest\n" }, { "path": "fuse/fusetest/helper.go", "content": "package fusetest\n\nimport (\n\t\"os\"\n\t\"os/exec\"\n)\n\n// LaunchAsProcess will start the fusemock server in another process.\n// This will serve a fuse mount on the specified directory and will listen\n// to commands on a http socket.\n//\n// NOTE: This will only work if you call this from a part of the main brig\n// executable. This relies on the executable to do start the server\n// when being called as »$0 debug fusemock«. Therefore this will not\n// work in tests, but it should be easy to adapt.\n//\n// The returned process can be used to terminate the program.\n// You should use the provided Dial() / Quit method to cleanup though.\nfunc LaunchAsProcess(opts Options) (*os.Process, error) {\n\tmyself, err := os.Executable()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\targs := []string{\n\t\t\"debug\",\n\t\t\"fusemock\",\n\t\t\"--mount-path\", opts.MountPath,\n\t\t\"--catfs-path\", opts.CatfsPath,\n\t\t\"--ipfs-path-or-multiaddr\", opts.IpfsPathOrMultiaddr,\n\t\t\"--url\", opts.URL,\n\t}\n\n\tif opts.MountReadOnly {\n\t\targs = append(args, \"--mount-ro\")\n\t}\n\n\tif opts.MountOffline {\n\t\targs = append(args, \"--mount-offline\")\n\t}\n\n\tcmd := exec.Command(myself, args...)\n\tif err := cmd.Start(); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn cmd.Process, nil\n}\n" }, { "path": "fuse/fusetest/server.go", "content": "package fusetest\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"net\"\n\t\"net/http\"\n\t\"os\"\n\t\"os/signal\"\n\t\"syscall\"\n\t\"time\"\n\n\tlog \"github.com/sirupsen/logrus\"\n\n\t\"github.com/gorilla/mux\"\n\t\"github.com/sahib/brig/backend/httpipfs\"\n\t\"github.com/sahib/brig/catfs\"\n\t\"github.com/sahib/brig/catfs/mio/pagecache/mdcache\"\n\t\"github.com/sahib/brig/defaults\"\n\t\"github.com/sahib/brig/fuse\"\n\t\"github.com/sahib/brig/repo/hints\"\n\t\"github.com/sahib/brig/util\"\n\t\"github.com/sahib/config\"\n)\n\nfunc makeFS(dbPath string, backend catfs.FsBackend) (*catfs.FS, error) {\n\t// open a dummy default config:\n\tcfg, err := config.Open(nil, defaults.Defaults, config.StrictnessPanic)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\thintMgr, err := hints.NewManager(nil)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tpageCache, err := mdcache.New(mdcache.Options{\n\t\tMaxMemoryUsage: 1024 * 1024 * 1024,\n\t\tL1CacheMissRefill: true,\n\t})\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcfs, err := catfs.NewFilesystem(\n\t\tbackend,\n\t\tdbPath,\n\t\t\"alice\",\n\t\tfalse,\n\t\tcfg.Section(\"fs\"),\n\t\thintMgr,\n\t\tpageCache,\n\t)\n\n\tif err != nil {\n\t\tlog.Fatalf(\"Failed to create catfs filesystem: %v\", err)\n\t\treturn nil, err\n\t}\n\n\treturn cfs, err\n}\n\nfunc mount(cfs *catfs.FS, mountPath string, opts Options) (*fuse.Mount, error) {\n\tif err := os.MkdirAll(mountPath, 0700); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn fuse.NewMount(cfs, mountPath, nil, fuse.MountOptions{\n\t\tReadOnly: opts.MountReadOnly,\n\t\tOffline: opts.MountOffline,\n\t\tRoot: \"/\",\n\t})\n}\n\nfunc serveHTTPServer(opts Options) error {\n\tscheme, addr, err := util.URLToSchemeAndAddr(opts.URL)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tlst, err := net.Listen(scheme, addr)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Needed for /quit.\n\tsrv := &http.Server{}\n\n\tsigCh := make(chan os.Signal, 1)\n\tsignal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM)\n\n\t// Properly exit when Ctrl-C is pressed.\n\t// (including unmounting!)\n\tgo func() {\n\t\t<-sigCh\n\n\t\tctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)\n\t\tdefer cancel()\n\n\t\tsrv.Shutdown(ctx)\n\t}()\n\n\t// TODO: routes for stage / cat (although not really necessary...)\n\t// TODO: root for unmount.\n\trouter := mux.NewRouter()\n\trouter.HandleFunc(\"/quit\", func(w http.ResponseWriter, r *http.Request) {\n\t\tgo func() {\n\t\t\t// Close the server in a few ms, just not in th request itself.\n\t\t\t// Otherwise the client will block forever.\n\t\t\ttime.Sleep(100 * time.Millisecond)\n\t\t\tif err := srv.Shutdown(r.Context()); err != nil {\n\t\t\t\tlog.WithError(err).Warnf(\"failed to shutdown server\")\n\t\t\t}\n\t\t}()\n\t}).Methods(\"GET\")\n\n\tsrv.Handler = router\n\tfmt.Println(\"serving...\")\n\tdefer fmt.Println(\"serving done...\")\n\treturn srv.Serve(lst)\n}\n\n// Options can be specified to control the behavior of the fusetest server.\ntype Options struct {\n\t// MountPath is where the fuse mount will be available.\n\tMountPath string\n\n\t// CatfsPath is where the metdata is stored.\n\tCatfsPath string\n\n\t// IpfsPath tells us which IPFS repo to use.\n\t// If empty, use the mock backend.\n\tIpfsPathOrMultiaddr string\n\n\t// URL defines where the server can be reached.\n\tURL string\n\n\t// MountReadOnly = true means to not allow modifications.\n\tMountReadOnly bool\n\n\t// MountOffline= true means to not allow online queries.\n\tMountOffline bool\n}\n\n// Launch will launch a fuse test server.\nfunc Launch(opts Options) error {\n\ttmpDir, err := ioutil.TempDir(\"\", \"brig-debug-fuse-*\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdefer os.RemoveAll(tmpDir)\n\n\tfor _, path := range []string{opts.MountPath, opts.CatfsPath} {\n\t\tif err := os.MkdirAll(path, 0700); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tvar backend catfs.FsBackend\n\tif opts.IpfsPathOrMultiaddr != \"\" {\n\t\tbackend, err = httpipfs.NewNode(opts.IpfsPathOrMultiaddr, \"\")\n\t} else {\n\t\tbackend = catfs.NewMemFsBackend()\n\t}\n\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tcfs, err := makeFS(opts.CatfsPath, backend)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tm, err := mount(cfs, opts.MountPath, opts)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// make sure it gets closed, even when no unmount is happening.\n\tdefer func() {\n\t\tfmt.Println(\"Closing mount\")\n\t\tif err := m.Close(); err != nil {\n\t\t\tlog.WithError(err).Error(\"fuse mount close failed\")\n\t\t}\n\t}()\n\n\treturn serveHTTPServer(opts)\n}\n" }, { "path": "fuse/handle.go", "content": "// +build !windows\n\npackage fuse\n\nimport (\n\t\"io\"\n\t\"sync\"\n\t\"syscall\"\n\t\"time\"\n\n\t\"context\"\n\n\t\"bazil.org/fuse\"\n\t\"bazil.org/fuse/fs\"\n\t\"github.com/sahib/brig/catfs\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// Handle is an open Entry.\ntype Handle struct {\n\tmu sync.Mutex\n\tfd *catfs.Handle\n\tm *Mount\n\twasModified bool\n}\n\n// Read is called to read a block of data at a certain offset.\nfunc (hd *Handle) Read(ctx context.Context, req *fuse.ReadRequest, resp *fuse.ReadResponse) error {\n\thd.mu.Lock()\n\tdefer hd.mu.Unlock()\n\tdefer logPanic(\"handle: read\")\n\n\tlog.Debugf(\n\t\t\"fuse-Read: %s (off: %d size: %d)\",\n\t\thd.fd.Path(),\n\t\treq.Offset,\n\t\treq.Size,\n\t)\n\n\tn, err := hd.fd.ReadAt(resp.Data[:req.Size], req.Offset)\n\tif err != nil && err != io.EOF {\n\t\treturn errorize(\"handle-read-io\", err)\n\t}\n\n\tresp.Data = resp.Data[:n]\n\treturn nil\n}\n\n// Write is called to write a block of data at a certain offset.\n// Note: do not assume that Write requests come in `fifo` order from the OS level!!!\n// I.e. during `cp largeFile /brig-fuse-mount/newFile`\n// the kernel might occasionally send write requests with blocks out of order!!!\n// In other words stream-like optimizations are not possible .\nfunc (hd *Handle) Write(ctx context.Context, req *fuse.WriteRequest, resp *fuse.WriteResponse) error {\n\tstart := time.Now()\n\thd.mu.Lock()\n\tdefer hd.mu.Unlock()\n\tdefer logPanic(\"handle: write\")\n\n\tlog.Debugf(\n\t\t\"fuse-Write: %s (off: %d size: %d)\",\n\t\thd.fd.Path(),\n\t\treq.Offset,\n\t\tlen(req.Data),\n\t)\n\n\t// Offset seems to be always provided from the start (i.e. 0)\n\tn, err := hd.writeAt(req.Data, req.Offset)\n\tresp.Size = n\n\tif err != nil {\n\t\treturn errorize(\"handle-write-io\", err)\n\t}\n\tif n != len(req.Data) {\n\t\tlog.Panicf(\"written amount %d is not equal to requested %d\", n, len(req.Data))\n\t\treturn err\n\t}\n\tlog.Infof(\"fuse: Write time %v for %d bytes\", time.Since(start), n)\n\thd.wasModified = true\n\treturn nil\n}\n\n// Writes data from `buf` at offset `off` counted from the start (0 offset).\n// Mimics `WriteAt` from `io` package https://golang.org/pkg/io/#WriterAt\n// Main idea is not bother with Seek pointer, since underlying `overlay` works\n// with intervals in memory and we do not need to `Seek` the backend which is very time expensive.\nfunc (hd *Handle) writeAt(buf []byte, off int64) (n int, err error) {\n\tn, err = hd.fd.WriteAt(buf, off)\n\tif n != len(buf) || err != nil {\n\t\tlog.Errorf(\"fuse: were not able to save %d bytes at offset %d\", len(buf), off)\n\t}\n\treturn n, err\n}\n\n// Flush is called to make sure all written contents get synced to disk.\nfunc (hd *Handle) Flush(ctx context.Context, req *fuse.FlushRequest) error {\n\treturn hd.flush()\n}\n\n// flush does the actual adding to brig.\nfunc (hd *Handle) flush() error {\n\thd.mu.Lock()\n\tdefer hd.mu.Unlock()\n\n\tlog.Debugf(\"fuse-flush: %v\", hd.fd.Path())\n\tdefer logPanic(\"handle: flush\")\n\n\tif !hd.wasModified {\n\t\treturn nil\n\t}\n\tstart := time.Now()\n\tif err := hd.fd.Flush(); err != nil {\n\t\treturn errorize(\"handle-flush\", err)\n\t}\n\tlog.Infof(\"fuse: Flashed `%s` in %v\", hd.fd.Path(), time.Since(start))\n\thd.wasModified = false\n\n\tnotifyChange(hd.m, 500*time.Millisecond)\n\treturn nil\n}\n\n// Release is called to close this handle.\nfunc (hd *Handle) Release(ctx context.Context, req *fuse.ReleaseRequest) error {\n\tdefer logPanic(\"handle: release\")\n\tlog.Debugf(\"fuse-release: %v\", hd.fd.Path())\n\n\tif req.Flags.IsReadOnly() {\n\t\t// we don't need to track read-only handles\n\t\treturn nil\n\t}\n\n\terr := hd.flush()\n\tif err != nil {\n\t\treturn errorize(\"handle-release\", err)\n\t}\n\n\treturn nil\n}\n\n// Truncates (or extends) data to the desired size\nfunc (hd *Handle) truncate(size uint64) error {\n\tlog.Debugf(\"fuse-truncate: %v to size %d\", hd.fd.Path(), size)\n\tdefer logPanic(\"handle: truncate\")\n\terr := hd.fd.Truncate(size)\n\n\treturn err\n}\n\n// Poll checks that the handle is ready for I/O or not\nfunc (hd *Handle) Poll(ctx context.Context, req *fuse.PollRequest, resp *fuse.PollResponse) error {\n\t// Comment taken verbatim from fs/serve.go of bazil.org/fuse:\n\t// Poll checks whether the handle is currently ready for I/O, and\n\t// may request a wakeup when it is.\n\t//\n\t// Poll should always return quickly. Clients waiting for\n\t// readiness can be woken up by passing the return value of\n\t// PollRequest.Wakeup to fs.Server.NotifyPollWakeup or\n\t// fuse.Conn.NotifyPollWakeup.\n\t//\n\t// To allow supporting poll for only some of your Nodes/Handles,\n\t// the default behavior is to report immediate readiness. If your\n\t// FS does not support polling and you want to minimize needless\n\t// requests and log noise, implement NodePoller and return\n\t// syscall.ENOSYS.\n\t//\n\t// The Go runtime uses epoll-based I/O whenever possible, even for\n\t// regular files.\n\n\t// Here we implement a dummy response which reports \"I am ready\".\n\t// The access separation is handled by mutex, so go-rutines\n\t// will have to be blocked but its ok. We do not expect many\n\t// processes working with the same file\n\n\t// default always ready mask\n\tresp.REvents = fuse.DefaultPollMask\n\n\t// We also return ENOSYS error, which sort of invalidate our response,\n\t// the ENOSYS indicates that this call is not supported\n\treturn syscall.ENOSYS\n}\n\n// Compiler checks to see if we got all the interfaces right:\nvar _ = fs.HandleFlusher(&Handle{})\nvar _ = fs.HandleReader(&Handle{})\nvar _ = fs.HandleReleaser(&Handle{})\nvar _ = fs.HandleWriter(&Handle{})\nvar _ = fs.HandlePoller(&Handle{})\n" }, { "path": "fuse/mount.go", "content": "// +build !windows\n\npackage fuse\n\nimport (\n\t\"bytes\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"os/exec\"\n\t\"path\"\n\t\"sync\"\n\t\"time\"\n\n\t\"bazil.org/fuse\"\n\t\"bazil.org/fuse/fs\"\n\te \"github.com/pkg/errors\"\n\t\"github.com/sahib/brig/catfs\"\n\t\"github.com/sahib/brig/util\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// Notifier implementors can take notifications\n// from any events happening in the fuse mount.\ntype Notifier interface {\n\t// PublishEvent is called whenever a modification happens.\n\tPublishEvent()\n}\n\n// MountOptions defines all possible knobs you can turn for a mount.\n// The zero value are the default options.\ntype MountOptions struct {\n\t// ReadOnly makes the mount not modifyable\n\tReadOnly bool\n\t// Root determines what the root directory is.\n\tRoot string\n\t// Offline tells the mount to error out on files that would need\n\t// to be fetched from far.\n\tOffline bool\n}\n\n// This is very similar (and indeed mostly copied) code from:\n// https://github.com/bazil/fuse/blob/master/fs/fstestutil/mounted.go\n// Since that's \"only\" test module, api might change, so better have this\n// code here (also we might do a few things differently).\n\n// Mount represents a fuse endpoint on the filesystem.\n// It is used as top-level API to control a brigfs fuse mount.\ntype Mount struct {\n\tDir string\n\n\tfilesys *Filesystem\n\tclosed bool\n\tdone chan util.Empty\n\terrors chan error\n\tconn *fuse.Conn\n\tserver *fs.Server\n\toptions MountOptions\n\tnotifier Notifier\n\tfs *catfs.FS\n}\n\n// NewMount mounts a fuse endpoint at `mountpoint` retrieving data from `store`.\nfunc NewMount(cfs *catfs.FS, mountpoint string, notifier Notifier, opts MountOptions) (*Mount, error) {\n\tmountOptions := []fuse.MountOption{\n\t\tfuse.FSName(\"brigfs\"),\n\t\tfuse.Subtype(\"brig\"),\n\t\tfuse.AllowNonEmptyMount(),\n\t\t// enabling MaxReadahead double or even triple Read throughput 12MB/s -> 25 or 33 MB/s\n\t\tfuse.MaxReadahead(128 * 1024), // kernel uses at max 128kB = 131072B\n\t\t// enabling WritebackCache doubles write speed to buffer 12MB/s -> 24MB/s\n\t\tfuse.WritebackCache(), // writes will happen in mach large blocks 128kB instead of 8kB\n\t}\n\n\tif opts.ReadOnly {\n\t\tmountOptions = append(mountOptions, fuse.ReadOnly())\n\t}\n\n\tconn, err := fuse.Mount(mountpoint, mountOptions...)\n\tif err != nil {\n\t\treturn nil, e.Wrapf(err, \"fuse-mount\")\n\t}\n\n\tif opts.Root == \"\" {\n\t\topts.Root = \"/\"\n\t}\n\n\tinfo, err := cfs.Stat(opts.Root)\n\tif err != nil {\n\t\treturn nil, e.Wrapf(err, \"failed to lookup root node of mount: %v\", mountpoint)\n\t}\n\n\tif !info.IsDir {\n\t\treturn nil, e.Wrapf(err, \"%s is not a directory\", opts.Root)\n\t}\n\n\tmnt := &Mount{\n\t\tconn: conn,\n\t\tserver: fs.New(conn, nil),\n\t\tDir: mountpoint,\n\t\tdone: make(chan util.Empty),\n\t\terrors: make(chan error),\n\t\toptions: opts,\n\t\tnotifier: notifier,\n\t\tfs: cfs,\n\t}\n\tfilesys := &Filesystem{m: mnt, root: opts.Root}\n\tmnt.filesys = filesys\n\n\tgo func() {\n\t\tdefer close(mnt.done)\n\t\tlog.Debugf(\"serving fuse mount at %v\", mountpoint)\n\t\tmnt.errors <- mnt.server.Serve(filesys)\n\t\tmnt.done <- util.Empty{}\n\t\tlog.Debugf(\"stopped serving fuse at %v\", mountpoint)\n\t}()\n\n\tselect {\n\tcase <-mnt.conn.Ready:\n\t\tif err := mnt.conn.MountError; err != nil {\n\t\t\treturn nil, err\n\t\t}\n\tcase err = <-mnt.errors:\n\t\t// Serve quit early\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\treturn nil, errors.New(\"Serve exited early\")\n\t}\n\n\treturn mnt, nil\n}\n\nfunc lazyUnmount(dir string) error {\n\tcmd := exec.Command(\"fusermount\", \"-u\", \"-z\", dir) // #nosec\n\toutput, err := cmd.CombinedOutput()\n\tif err != nil {\n\t\tif len(output) > 0 {\n\t\t\toutput = bytes.TrimRight(output, \"\\n\")\n\t\t\tmsg := err.Error() + \": \" + string(output)\n\t\t\terr = errors.New(msg)\n\t\t}\n\t\treturn err\n\t}\n\treturn nil\n}\n\n// EqualOptions returns true when the options in `opts` have the same\n// option as currently set in the mount. If so, no re-mount is required.\nfunc (m *Mount) EqualOptions(opts MountOptions) bool {\n\tif m.options.ReadOnly != opts.ReadOnly {\n\t\treturn false\n\t}\n\n\treturn path.Clean(m.options.Root) == path.Clean(opts.Root)\n}\n\n// Close will wait until all I/O operations are done and unmount the fuse\n// mount again.\nfunc (m *Mount) Close() error {\n\tif m.closed {\n\t\treturn nil\n\t}\n\tm.closed = true\n\n\tlog.Infof(\"unmounting fuse mount at %v (this might take a bit)\", m.Dir)\n\n\tcouldUnmount := false\n\twaitTimeout := 1 * time.Second\n\n\t// Attempt unmounting several times:\n\tfor tries := 0; tries < 10; tries++ {\n\t\tif err := fuse.Unmount(m.Dir); err != nil {\n\t\t\tlog.Debugf(\"failed to graceful unmount: %v\", err)\n\t\t\ttime.Sleep(250 * time.Millisecond)\n\t\t\tcontinue\n\t\t}\n\n\t\tcouldUnmount = true\n\t\twaitTimeout = 5 * time.Second\n\t\tbreak\n\t}\n\n\tif !couldUnmount {\n\t\tlog.Warn(\"cant properly unmount; are there still processes using the mount?\")\n\t\tlog.Warn(\"attempting lazy umount (you might leak resources!)\")\n\t\tif err := lazyUnmount(m.Dir); err != nil {\n\t\t\tlog.Debugf(\"lazy unmount failed: %v\", err)\n\t\t}\n\t}\n\n\t// Be sure to drain the error channel:\n\tselect {\n\tcase err := <-m.errors:\n\t\tif err != nil {\n\t\t\tlog.Warningf(\"fuse returned an error: %v\", err)\n\t\t}\n\tcase <-time.NewTimer(waitTimeout).C:\n\t\t// blocking due to fuse freeze.\n\t}\n\n\t// Be sure to pull the item from the channel:\n\tselect {\n\tcase <-m.done:\n\t\tlog.Debugf(\"gracefully shutting down\")\n\tcase <-time.NewTimer(waitTimeout).C:\n\t\t// success or blocking due to fuse freeze.\n\t}\n\n\t// If we could not unmount, schedule closing in the background.\n\t// This might be leaky, since Close might not ever return.\n\t// But usually we unmount on program exit anyways...\n\tif couldUnmount {\n\t\tif err := m.conn.Close(); err != nil {\n\t\t\treturn err\n\t\t}\n\t} else {\n\t\tgo m.conn.Close()\n\t}\n\n\treturn nil\n}\n\n// MountTable is a mapping from the mountpoint to the respective\n// `Mount` struct. It's given as convenient way to maintain several mounts.\n// All operations on the table are safe to call from several goroutines.\ntype MountTable struct {\n\tmu sync.Mutex\n\tm map[string]*Mount\n\tfs *catfs.FS\n\tnotifier Notifier\n}\n\n// NewMountTable returns an empty mount table.\nfunc NewMountTable(fs *catfs.FS, notifier Notifier) *MountTable {\n\treturn &MountTable{\n\t\tm: make(map[string]*Mount),\n\t\tfs: fs,\n\t\tnotifier: notifier,\n\t}\n}\n\n// AddMount calls NewMount and adds it to the table at `path`.\nfunc (t *MountTable) AddMount(path string, opts MountOptions) (*Mount, error) {\n\tt.mu.Lock()\n\tdefer t.mu.Unlock()\n\n\treturn t.addMount(path, opts)\n}\n\nfunc checkMountPath(path string) error {\n\tfiles, err := ioutil.ReadDir(path)\n\tif err != nil {\n\t\t// This will also fail if `path` is not a directory:\n\t\treturn err\n\t}\n\n\tif len(files) > 0 {\n\t\treturn fmt.Errorf(\"Refusing to mount over non-empty dir `%s`\", path)\n\t}\n\n\treturn nil\n}\n\nfunc (t *MountTable) addMount(path string, opts MountOptions) (*Mount, error) {\n\tif err := checkMountPath(path); err != nil {\n\t\treturn nil, e.Wrapf(err, \"dir check\")\n\t}\n\n\tm, ok := t.m[path]\n\tif ok {\n\t\treturn m, nil\n\t}\n\n\tm, err := NewMount(t.fs, path, t.notifier, opts)\n\tif err == nil {\n\t\tt.m[path] = m\n\t}\n\n\treturn m, e.Wrapf(err, \"new-mount\")\n}\n\n// Unmount closes the mount at `path` and deletes it from the table.\nfunc (t *MountTable) Unmount(path string) error {\n\tt.mu.Lock()\n\tdefer t.mu.Unlock()\n\n\treturn t.unmount(path)\n}\n\nfunc (t *MountTable) unmount(path string) error {\n\tm, ok := t.m[path]\n\tif !ok {\n\t\treturn fmt.Errorf(\"no mount at `%v`\", path)\n\t}\n\n\tdelete(t.m, path)\n\treturn m.Close()\n}\n\n// Close unmounts all leftover mounts and clears the table.\nfunc (t *MountTable) Close() error {\n\tt.mu.Lock()\n\tdefer t.mu.Unlock()\n\n\tvar err error\n\n\tfor _, mount := range t.m {\n\t\tif closeErr := mount.Close(); closeErr != nil {\n\t\t\terr = closeErr\n\t\t}\n\t}\n\n\tt.m = make(map[string]*Mount)\n\treturn err\n}\n" }, { "path": "fuse/stub.go", "content": "// Package fuse implements a stub.\n// +build windows\n\n// The following comment is only here to make sure golint ignores this file:\n// Code generated by me DO NOT EDIT.\n\n// This package is intentend for platforms that do not offer fuse.\n// It rebuilds the same API as the rest of the package with stubs.\npackage fuse\n\nimport (\n\t\"errors\"\n\n\t\"github.com/sahib/brig/catfs\"\n\t\"github.com/sahib/config\"\n)\n\ntype Notifier interface {\n\tPublishEvent()\n}\n\nvar ErrCompiledWithoutFuse = errors.New(\"brig was compiled without fuse support\")\n\ntype MountOptions struct {\n\tReadOnly bool\n\tRoot string\n}\n\ntype Mount struct {\n\tDir string\n}\n\nfunc NewMount(cfs *catfs.FS, mountpoint string, opts MountOptions) (*Mount, error) {\n\treturn nil, ErrCompiledWithoutFuse\n}\n\nfunc (m *Mount) EqualOptions(opts MountOptions) bool {\n\treturn false\n}\n\nfunc (m *Mount) Close() error {\n\treturn ErrCompiledWithoutFuse\n}\n\ntype MountTable struct{}\n\nfunc NewMountTable(fs *catfs.FS, notifier Notifier) *MountTable {\n\treturn nil\n}\n\nfunc (t *MountTable) AddMount(path string, opts MountOptions) (*Mount, error) {\n\treturn nil, ErrCompiledWithoutFuse\n}\n\nfunc (t *MountTable) Unmount(path string) error {\n\treturn ErrCompiledWithoutFuse\n}\n\nfunc (t *MountTable) Close() error {\n\treturn ErrCompiledWithoutFuse\n}\n\ntype FsTabEntry struct {\n\tName string\n\tPath string\n\tRoot string\n\tActive bool\n\tReadOnly bool\n}\n\nfunc FsTabAdd(cfg *config.Config, name, path string, opts MountOptions) error {\n\treturn ErrCompiledWithoutFuse\n}\n\nfunc FsTabRemove(cfg *config.Config, name string) error {\n\treturn ErrCompiledWithoutFuse\n}\n\nfunc FsTabUnmountAll(cfg *config.Config, mounts *MountTable) error {\n\treturn ErrCompiledWithoutFuse\n}\n\nfunc FsTabApply(cfg *config.Config, mounts *MountTable) error {\n\treturn ErrCompiledWithoutFuse\n}\n\nfunc FsTabList(cfg *config.Config, mounts *MountTable) ([]FsTabEntry, error) {\n\treturn nil, ErrCompiledWithoutFuse\n}\n" }, { "path": "fuse/util.go", "content": "// +build !windows\n\npackage fuse\n\nimport (\n\t\"fmt\"\n\t\"time\"\n\n\t\"bazil.org/fuse\"\n\t\"github.com/sahib/brig/catfs\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\t\"github.com/sahib/brig/repo/hints\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\nfunc errorize(name string, err error) error {\n\tif ie.IsNoSuchFileError(err) {\n\t\tlog.Infof(\"errorize: %s: No such file: %v\", name, err)\n\t\treturn fuse.ENOENT\n\t}\n\n\tif err != nil {\n\t\tlog.Warningf(\"fuse: %s: %v\", name, err)\n\t\treturn fuse.EIO\n\t}\n\n\treturn nil\n}\n\n// logPanic logs any panics by being called in a defer.\n// A rather inconvenient behaviour of fuse is to not report panics.\nfunc logPanic(name string) {\n\tif err := recover(); err != nil {\n\t\tlog.Errorf(\"bug: %s panicked: %v\", name, err)\n\t}\n}\n\ntype xattrHandler struct {\n\tget func(cfs *catfs.FS, info *catfs.StatInfo) ([]byte, error)\n\tset func(cfs *catfs.FS, path string, value []byte) error\n}\n\nvar xattrMap = map[string]xattrHandler{\n\t\"user.brig.hash.content\": {\n\t\tget: func(cfs *catfs.FS, info *catfs.StatInfo) ([]byte, error) {\n\t\t\treturn []byte(info.ContentHash.B58String()), nil\n\t\t},\n\t},\n\t\"user.brig.hash.tree\": {\n\t\tget: func(cfs *catfs.FS, info *catfs.StatInfo) ([]byte, error) {\n\t\t\treturn []byte(info.TreeHash.B58String()), nil\n\t\t},\n\t},\n\t\"user.brig.hash.backend\": {\n\t\tget: func(cfs *catfs.FS, info *catfs.StatInfo) ([]byte, error) {\n\t\t\treturn []byte(info.BackendHash.B58String()), nil\n\t\t},\n\t},\n\t\"user.brig.pinned\": {\n\t\tget: func(cfs *catfs.FS, info *catfs.StatInfo) ([]byte, error) {\n\t\t\tif info.IsPinned {\n\t\t\t\treturn []byte(\"yes\"), nil\n\t\t\t}\n\t\t\treturn []byte(\"no\"), nil\n\t\t},\n\t},\n\t\"user.brig.explicitly_pinned\": {\n\t\tget: func(cfs *catfs.FS, info *catfs.StatInfo) ([]byte, error) {\n\t\t\tif info.IsExplicit {\n\t\t\t\treturn []byte(\"yes\"), nil\n\t\t\t}\n\t\t\treturn []byte(\"no\"), nil\n\t\t},\n\t},\n\t\"user.brig.hints.encryption\": {\n\t\tget: func(cfs *catfs.FS, info *catfs.StatInfo) ([]byte, error) {\n\t\t\treturn []byte(cfs.Hints().Lookup(info.Path).EncryptionAlgo), nil\n\t\t},\n\t\tset: func(cfs *catfs.FS, path string, val []byte) error {\n\t\t\thint := cfs.Hints().Lookup(path)\n\t\t\thint.EncryptionAlgo = hints.EncryptionHint(val)\n\t\t\tif !hint.IsValid() {\n\t\t\t\treturn fmt.Errorf(\"bad encryption algorithm: %s\", string(val))\n\t\t\t}\n\n\t\t\treturn cfs.Hints().Set(path, hint)\n\t\t},\n\t},\n\t\"user.brig.hints.compression\": {\n\t\tget: func(cfs *catfs.FS, info *catfs.StatInfo) ([]byte, error) {\n\t\t\treturn []byte(cfs.Hints().Lookup(info.Path).CompressionAlgo), nil\n\t\t},\n\t\tset: func(cfs *catfs.FS, path string, val []byte) error {\n\t\t\thint := cfs.Hints().Lookup(path)\n\t\t\thint.CompressionAlgo = hints.CompressionHint(val)\n\t\t\tif !hint.IsValid() {\n\t\t\t\treturn fmt.Errorf(\"bad compression algorithm: %s\", string(val))\n\t\t\t}\n\n\t\t\treturn cfs.Hints().Set(path, hint)\n\t\t},\n\t},\n}\n\nfunc listXattr() []byte {\n\tresp := []byte{}\n\tfor k := range xattrMap {\n\t\tresp = append(resp, k...)\n\t\tresp = append(resp, '\\x00')\n\t}\n\n\treturn resp\n}\n\nfunc getXattr(cfs *catfs.FS, name, path string) ([]byte, error) {\n\thandler, ok := xattrMap[name]\n\tif !ok || handler.get == nil {\n\t\treturn nil, fuse.ErrNoXattr\n\t}\n\n\tinfo, err := cfs.Stat(path)\n\tif err != nil {\n\t\treturn nil, errorize(\"getxattr\", err)\n\t}\n\n\treturn handler.get(cfs, info)\n}\n\nfunc setXattr(cfs *catfs.FS, name, path string, val []byte) error {\n\thandler, ok := xattrMap[name]\n\tif !ok || handler.set == nil {\n\t\treturn fuse.ErrNoXattr\n\t}\n\n\tif err := handler.set(cfs, path, val); err != nil {\n\t\treturn fuse.EIO\n\t}\n\n\treturn nil\n}\n\nfunc notifyChange(m *Mount, d time.Duration) {\n\tif m.notifier == nil {\n\t\t// this can happen in tests.\n\t\treturn\n\t}\n\n\ttime.AfterFunc(d, m.notifier.PublishEvent)\n}\n" }, { "path": "gateway/db/capnp/user.capnp", "content": "using Go = import \"/go.capnp\";\n\n@0xa0b1c18bd0f965c4;\n\n$Go.package(\"capnp\");\n$Go.import(\"github.com/sahib/brig/gateway/db/capnp\");\n\nstruct User {\n\tname @0 :Text;\n\tpasswordHash @1 :Text;\n\tsalt @2 :Text;\n\tfolders @3 :List(Text);\n\trights @4 :List(Text);\n}\n" }, { "path": "gateway/db/capnp/user.capnp.go", "content": "// Code generated by capnpc-go. DO NOT EDIT.\n\npackage capnp\n\nimport (\n\tcapnp \"zombiezen.com/go/capnproto2\"\n\ttext \"zombiezen.com/go/capnproto2/encoding/text\"\n\tschemas \"zombiezen.com/go/capnproto2/schemas\"\n)\n\ntype User struct{ capnp.Struct }\n\n// User_TypeID is the unique identifier for the type User.\nconst User_TypeID = 0x861de4463c5a4a22\n\nfunc NewUser(s *capnp.Segment) (User, error) {\n\tst, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 5})\n\treturn User{st}, err\n}\n\nfunc NewRootUser(s *capnp.Segment) (User, error) {\n\tst, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 5})\n\treturn User{st}, err\n}\n\nfunc ReadRootUser(msg *capnp.Message) (User, error) {\n\troot, err := msg.RootPtr()\n\treturn User{root.Struct()}, err\n}\n\nfunc (s User) String() string {\n\tstr, _ := text.Marshal(0x861de4463c5a4a22, s.Struct)\n\treturn str\n}\n\nfunc (s User) Name() (string, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.Text(), err\n}\n\nfunc (s User) HasName() bool {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s User) NameBytes() ([]byte, error) {\n\tp, err := s.Struct.Ptr(0)\n\treturn p.TextBytes(), err\n}\n\nfunc (s User) SetName(v string) error {\n\treturn s.Struct.SetText(0, v)\n}\n\nfunc (s User) PasswordHash() (string, error) {\n\tp, err := s.Struct.Ptr(1)\n\treturn p.Text(), err\n}\n\nfunc (s User) HasPasswordHash() bool {\n\tp, err := s.Struct.Ptr(1)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s User) PasswordHashBytes() ([]byte, error) {\n\tp, err := s.Struct.Ptr(1)\n\treturn p.TextBytes(), err\n}\n\nfunc (s User) SetPasswordHash(v string) error {\n\treturn s.Struct.SetText(1, v)\n}\n\nfunc (s User) Salt() (string, error) {\n\tp, err := s.Struct.Ptr(2)\n\treturn p.Text(), err\n}\n\nfunc (s User) HasSalt() bool {\n\tp, err := s.Struct.Ptr(2)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s User) SaltBytes() ([]byte, error) {\n\tp, err := s.Struct.Ptr(2)\n\treturn p.TextBytes(), err\n}\n\nfunc (s User) SetSalt(v string) error {\n\treturn s.Struct.SetText(2, v)\n}\n\nfunc (s User) Folders() (capnp.TextList, error) {\n\tp, err := s.Struct.Ptr(3)\n\treturn capnp.TextList{List: p.List()}, err\n}\n\nfunc (s User) HasFolders() bool {\n\tp, err := s.Struct.Ptr(3)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s User) SetFolders(v capnp.TextList) error {\n\treturn s.Struct.SetPtr(3, v.List.ToPtr())\n}\n\n// NewFolders sets the folders field to a newly\n// allocated capnp.TextList, preferring placement in s's segment.\nfunc (s User) NewFolders(n int32) (capnp.TextList, error) {\n\tl, err := capnp.NewTextList(s.Struct.Segment(), n)\n\tif err != nil {\n\t\treturn capnp.TextList{}, err\n\t}\n\terr = s.Struct.SetPtr(3, l.List.ToPtr())\n\treturn l, err\n}\n\nfunc (s User) Rights() (capnp.TextList, error) {\n\tp, err := s.Struct.Ptr(4)\n\treturn capnp.TextList{List: p.List()}, err\n}\n\nfunc (s User) HasRights() bool {\n\tp, err := s.Struct.Ptr(4)\n\treturn p.IsValid() || err != nil\n}\n\nfunc (s User) SetRights(v capnp.TextList) error {\n\treturn s.Struct.SetPtr(4, v.List.ToPtr())\n}\n\n// NewRights sets the rights field to a newly\n// allocated capnp.TextList, preferring placement in s's segment.\nfunc (s User) NewRights(n int32) (capnp.TextList, error) {\n\tl, err := capnp.NewTextList(s.Struct.Segment(), n)\n\tif err != nil {\n\t\treturn capnp.TextList{}, err\n\t}\n\terr = s.Struct.SetPtr(4, l.List.ToPtr())\n\treturn l, err\n}\n\n// User_List is a list of User.\ntype User_List struct{ capnp.List }\n\n// NewUser creates a new list of User.\nfunc NewUser_List(s *capnp.Segment, sz int32) (User_List, error) {\n\tl, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 0, PointerCount: 5}, sz)\n\treturn User_List{l}, err\n}\n\nfunc (s User_List) At(i int) User { return User{s.List.Struct(i)} }\n\nfunc (s User_List) Set(i int, v User) error { return s.List.SetStruct(i, v.Struct) }\n\nfunc (s User_List) String() string {\n\tstr, _ := text.MarshalList(0x861de4463c5a4a22, s.List)\n\treturn str\n}\n\n// User_Promise is a wrapper for a User promised by a client call.\ntype User_Promise struct{ *capnp.Pipeline }\n\nfunc (p User_Promise) Struct() (User, error) {\n\ts, err := p.Pipeline.Struct()\n\treturn User{s}, err\n}\n\nconst schema_a0b1c18bd0f965c4 = \"x\\xda\\\\\\xca1J\\x03A\\x18\\xc5\\xf1\\xf7ff\\x15$\" +\n\t\"$\\x0e\\xac\\x9d\\\"\\x0a6\\x16\\x06\\xdb (\\x16\\\"V\\xf9\" +\n\t\"\\x0a\\x1b\\x1b\\x19\\xdd11\\xc4d\\xd9Y\\x89\\x16\\xa2B\\x10\" +\n\t\"E\\x05\\x0f`\\xa1\\xe0\\x05\\xecl\\x05\\x0f\\xe0!\\xbc\\x84\\xd5\" +\n\t\"\\xca\\x06\\xd2\\xd8\\xfd\\xdf\\x8f7\\xdd\\xd9P6\\xda\\x07\\xc4D\" +\n\t\"\\x13\\xc5\\xe2\\xce\\xde\\xda\\xd6\\xcf\\xdc\\x0d\\xec,\\x8b/\\xff\\xfb\" +\n\t\"}\\xff\\xf9\\xfe\\x82(\\x9a\\x04Vg\\xa6h\\x97\\xcaX\\x98\" +\n\t\"'X\\xb4\\\\\\xee\\x07\\xee\\xbc\\xae\\x93\\x83\\xfa\\xa1K{i\" +\n\t\"\\xfd4\\xf8le\\x94\\x8d\\xdd\\xe03\\xa0IJ\\xac\\x0d`\" +\n\t\"\\x08\\xd8\\x8be@\\xce4e\\xa8h\\xc9\\x98%^w\\x00\" +\n\t\"\\xb9\\xd2\\x94\\x07E\\xabTL\\x05\\xd8\\xbb\\xf29\\xd4\\x94'\" +\n\t\"E\\xabuL\\x0d\\xd8\\xc7M@n5\\xe5M\\xd1\\x1a\\x13\" +\n\t\"\\xd3\\x00\\xf6\\xb5\\x01\\xc8\\xb3\\xa6|(\\xd6z\\xee\\xc4\\xb3\\x02\" +\n\t\"\\xc5\\x0aX\\xa4.\\x84A?KP\\xdbv\\xa1=\\xe6Z\" +\n\t\"p\\xdd|<.\\x8f\\xfa\\xdd\\xc4g\\x81U\\xb0\\xa99\\xe2\" +\n\t\"*\\xb8\\x9e\\x1d\\xb7\\xda\\xf9\\x7f\\xfd\\x0b\\x00\\x00\\xff\\xff\\x84\\x94\" +\n\t\":\\x18\"\n\nfunc init() {\n\tschemas.Register(schema_a0b1c18bd0f965c4,\n\t\t0x861de4463c5a4a22)\n}\n" }, { "path": "gateway/db/db.go", "content": "package db\n\nimport (\n\t\"crypto/rand\"\n\t\"crypto/subtle\"\n\t\"encoding/base64\"\n\t\"fmt\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"time\"\n\n\tbadger \"github.com/dgraph-io/badger/v3\"\n\tcapnp \"github.com/sahib/brig/gateway/db/capnp\"\n\t\"github.com/sahib/brig/util\"\n\tlog \"github.com/sirupsen/logrus\"\n\tcapnp_lib \"zombiezen.com/go/capnproto2\"\n)\n\nconst (\n\t// RightDownload refers to the right to download/view a file.\n\tRightDownload = \"fs.download\"\n\t// RightFsView refers to the right to view everything related to\n\t// the filesystem and history.\n\tRightFsView = \"fs.view\"\n\t// RightFsEdit refers to the right to edit the filesystem.\n\t// This includes pinning.\n\tRightFsEdit = \"fs.edit\"\n\t// RightRemotesView is the right to view the remote list.\n\tRightRemotesView = \"remotes.view\"\n\t// RightRemotesEdit is the right to edit the remote list.\n\tRightRemotesEdit = \"remotes.edit\"\n)\n\nvar (\n\t// DefaultRights is a list of rights that users will get\n\t// if no other explicit rights are given. They are identical\n\t// to the admin role currently.\n\tDefaultRights = []string{\n\t\tRightDownload,\n\t\tRightFsView,\n\t\tRightFsEdit,\n\t\tRightRemotesView,\n\t\tRightRemotesEdit,\n\t}\n\n\t// AllRights is a map that can be quickly used to check\n\t// if a right is valid or not.\n\tAllRights = map[string]bool{\n\t\tRightDownload: true,\n\t\tRightFsView: true,\n\t\tRightFsEdit: true,\n\t\tRightRemotesView: true,\n\t\tRightRemotesEdit: true,\n\t}\n)\n\n// UserDatabase is a badger db that stores user information,\n// using the user name as unique key.\ntype UserDatabase struct {\n\tisStopped int64\n\tmu sync.Mutex\n\tdb *badger.DB\n\tgcTicker *time.Ticker\n}\n\n// NewUserDatabase creates a new UserDatabase at `path` or loads\n// an existing one.\nfunc NewUserDatabase(path string) (*UserDatabase, error) {\n\topts := badger.DefaultOptions(path).\n\t\tWithValueLogFileSize(10 * 1024 * 1024). //default is 2GB we should not need 2GB\n\t\tWithMemTableSize(10 * 1024 * 1024). //default is 64MB\n\t\tWithSyncWrites(false).\n\t\tWithLogger(nil)\n\n\tdb, err := badger.Open(opts)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tgcTicker := time.NewTicker(5 * time.Minute)\n\n\tudb := &UserDatabase{db: db, gcTicker: gcTicker}\n\n\tgo func() {\n\t\tfor range gcTicker.C {\n\t\t\tif atomic.LoadInt64(&udb.isStopped) > 0 {\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif err := db.RunValueLogGC(0.5); err != nil {\n\t\t\t\tif err != badger.ErrNoRewrite {\n\t\t\t\t\tlog.WithError(err).Warnf(\"badger gc failed\")\n\t\t\t\t}\n\t\t\t}\n\n\t\t}\n\t}()\n\n\treturn udb, nil\n}\n\n// Close cleans up all the resources used by a badger db.\nfunc (ub *UserDatabase) Close() error {\n\tub.mu.Lock()\n\tdefer ub.mu.Unlock()\n\n\tub.gcTicker.Stop()\n\tatomic.StoreInt64(&ub.isStopped, 1)\n\n\tif err := ub.db.Close(); err != nil {\n\t\treturn err\n\t}\n\n\tub.db = nil\n\treturn nil\n}\n\nfunc unmarshalUser(data []byte) (*User, error) {\n\tmsg, err := capnp_lib.Unmarshal(data)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapUser, err := capnp.ReadRootUser(msg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn UserFromCapnp(capUser)\n}\n\n// UserFromCapnp takes a capnp.user and returns a regular User from it.\nfunc UserFromCapnp(capUser capnp.User) (*User, error) {\n\tcapFolders, err := capUser.Folders()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfolders := []string{}\n\tfor idx := 0; idx < capFolders.Len(); idx++ {\n\t\tfolder, err := capFolders.At(idx)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tfolders = append(folders, folder)\n\t}\n\n\tcapRights, err := capUser.Rights()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\trights := []string{}\n\tfor idx := 0; idx < capRights.Len(); idx++ {\n\t\tright, err := capRights.At(idx)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\trights = append(rights, right)\n\t}\n\n\tname, err := capUser.Name()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tpasswordHash, err := capUser.PasswordHash()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tsalt, err := capUser.Salt()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &User{\n\t\tName: name,\n\t\tPasswordHash: passwordHash,\n\t\tSalt: salt,\n\t\tFolders: folders,\n\t\tRights: rights,\n\t}, nil\n}\n\nfunc marshalUser(user *User) ([]byte, error) {\n\tmsg, seg, err := capnp_lib.NewMessage(capnp_lib.SingleSegment(nil))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif _, err := UserToCapnp(user, seg); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn msg.Marshal()\n}\n\n// UserToCapnp converts a User to a capnp.User.\nfunc UserToCapnp(user *User, seg *capnp_lib.Segment) (*capnp.User, error) {\n\tcapUser, err := capnp.NewRootUser(seg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapFolders, err := capnp_lib.NewTextList(seg, int32(len(user.Folders)))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor idx, folder := range user.Folders {\n\t\tif err := capFolders.Set(idx, folder); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tif err := capUser.SetFolders(capFolders); err != nil {\n\t\treturn nil, err\n\t}\n\n\tcapRights, err := capnp_lib.NewTextList(seg, int32(len(user.Rights)))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor idx, right := range user.Rights {\n\t\tif err := capRights.Set(idx, right); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tif err := capUser.SetRights(capRights); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := capUser.SetName(user.Name); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := capUser.SetPasswordHash(user.PasswordHash); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := capUser.SetSalt(user.Salt); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &capUser, nil\n}\n\n// User is one user that is stored in the database.\n// The passwords are stored as scrypt hash with added salt.\ntype User struct {\n\tName string\n\tPasswordHash string\n\tSalt string\n\tFolders []string\n\tRights []string\n}\n\n// CheckPassword checks if `password` matches the stored one.\nfunc (u User) CheckPassword(password string) (bool, error) {\n\tsalt, err := base64.StdEncoding.DecodeString(u.Salt)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\toldHash, err := base64.StdEncoding.DecodeString(u.PasswordHash)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\tnewHash := util.DeriveKey([]byte(password), salt, 32)\n\treturn subtle.ConstantTimeCompare(oldHash, newHash) == 1, nil\n}\n\n// HashPassword creates a new hash and salt from a password.\nfunc HashPassword(password string) (string, string, error) {\n\t// Read a new salt from a random source.\n\t// 8 bytes are considered enough by the scrypt documentation.\n\tsalt := make([]byte, 8)\n\tif n, err := rand.Read(salt); err != nil {\n\t\treturn \"\", \"\", err\n\t} else if n != 8 {\n\t\treturn \"\", \"\", fmt.Errorf(\"did not read enough random bytes\")\n\t}\n\n\t// Derive the actual hash and encode it to base64.\n\thash := util.DeriveKey([]byte(password), salt, 32)\n\tencode := base64.StdEncoding.EncodeToString\n\treturn encode(hash), encode(salt), nil\n}\n\n// Add adds a new user to the database.\n// If the user exists already, it is overwritten.\nfunc (ub *UserDatabase) Add(name, password string, folders []string, rights []string) error {\n\tub.mu.Lock()\n\tdefer ub.mu.Unlock()\n\n\tif len(folders) == 0 {\n\t\tfolders = []string{\"/\"}\n\t}\n\n\tif len(rights) == 0 {\n\t\trights = DefaultRights\n\t}\n\n\tfor _, right := range rights {\n\t\tif !AllRights[right] {\n\t\t\treturn fmt.Errorf(\"invalid right: %s\", right)\n\t\t}\n\t}\n\n\thashed, salt, err := HashPassword(password)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tuser := &User{\n\t\tName: name,\n\t\tPasswordHash: hashed,\n\t\tSalt: salt,\n\t\tFolders: folders,\n\t\tRights: rights,\n\t}\n\n\tdata, err := marshalUser(user)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn ub.db.Update(func(txn *badger.Txn) error {\n\t\treturn txn.Set([]byte(name), data)\n\t})\n}\n\n// Get returns a User, if it exists. If it does not exist,\n// an error will be returned.\nfunc (ub *UserDatabase) Get(name string) (User, error) {\n\tub.mu.Lock()\n\tdefer ub.mu.Unlock()\n\n\tuser := User{}\n\treturn user, ub.db.View(func(txn *badger.Txn) error {\n\t\titem, err := txn.Get([]byte(name))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn item.Value(func(data []byte) error {\n\t\t\tdecUser, err := unmarshalUser(data)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tuser = *decUser\n\t\t\treturn nil\n\t\t})\n\t})\n}\n\n// Remove removes an existing user.\nfunc (ub *UserDatabase) Remove(name string) error {\n\tub.mu.Lock()\n\tdefer ub.mu.Unlock()\n\n\treturn ub.db.Update(func(txn *badger.Txn) error {\n\t\t// Make sure to error out if the key did not exist:\n\t\tif _, err := txn.Get([]byte(name)); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\treturn txn.Delete([]byte(name))\n\t})\n}\n\n// List returns all users currently in the database.\nfunc (ub *UserDatabase) List() ([]User, error) {\n\tub.mu.Lock()\n\tdefer ub.mu.Unlock()\n\n\tusers := []User{}\n\treturn users, ub.db.View(func(txn *badger.Txn) error {\n\t\titer := txn.NewIterator(badger.IteratorOptions{})\n\t\tdefer iter.Close()\n\n\t\tfor iter.Rewind(); iter.Valid(); iter.Next() {\n\t\t\terr := iter.Item().Value(func(data []byte) error {\n\t\t\t\tuser, err := unmarshalUser(data)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\n\t\t\t\tusers = append(users, *user)\n\t\t\t\treturn nil\n\t\t\t})\n\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t})\n}\n" }, { "path": "gateway/db/db_test.go", "content": "package db\n\nimport (\n\t\"io/ioutil\"\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc withDummyDb(t *testing.T, fn func(db *UserDatabase)) {\n\ttmpPath, err := ioutil.TempDir(\"\", \"brig-gw-userdb\")\n\trequire.Nil(t, err)\n\tdefer os.RemoveAll(tmpPath)\n\n\tuserDb, err := NewUserDatabase(tmpPath)\n\trequire.Nil(t, err)\n\n\tfn(userDb)\n\n\trequire.Nil(t, userDb.Close())\n}\n\nfunc TestAddGet(t *testing.T) {\n\twithDummyDb(t, func(db *UserDatabase) {\n\t\trequire.Nil(t, db.Add(\"hello\", \"world\", []string{\"/\"}, []string{\"fs.view\"}))\n\t\tuser, err := db.Get(\"hello\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, \"hello\", user.Name)\n\t\trequire.NotEmpty(t, user.PasswordHash)\n\t\trequire.NotEmpty(t, user.Salt)\n\t\trequire.Equal(t, []string{\"/\"}, user.Folders)\n\t\trequire.Equal(t, []string{\"fs.view\"}, user.Rights)\n\t})\n}\n" }, { "path": "gateway/elm/.gitignore", "content": "elm-stuff\n" }, { "path": "gateway/elm/Makefile", "content": "SOURCES=$(shell find . -iname '*.elm')\n\nall: build minify\n\nrelease: release-build minify\n\nbuild:\n\t@elm make ${SOURCES} --output ../static/js/app.js\n\nrelease-build:\n\t@elm make ${SOURCES} --output ../static/js/app.js --optimize\n\nminify:\n\tuglifyjs ../static/js/app.js --compress 'pure_funcs=\"F2,F3,F4,F5,F6,F7,F8,F9,A2,A3,A4,A5,A6,A7,A8,A9\",pure_getters,keep_fargs=false,unsafe_comps,unsafe' | uglifyjs --mangle --output=../static/js/app.min.js\n\tmv ../static/js/app.min.js ../static/js/app.js\n" }, { "path": "gateway/elm/elm.json", "content": "{\n \"type\": \"application\",\n \"source-directories\": [\n \"src\"\n ],\n \"elm-version\": \"0.19.0\",\n \"dependencies\": {\n \"direct\": {\n \"NoRedInk/elm-json-decode-pipeline\": \"1.0.0\",\n \"andrewMacmurray/elm-delay\": \"3.0.0\",\n \"basti1302/elm-human-readable-filesize\": \"1.2.0\",\n \"elm/browser\": \"1.0.0\",\n \"elm/core\": \"1.0.2\",\n \"elm/file\": \"1.0.1\",\n \"elm/html\": \"1.0.0\",\n \"elm/http\": \"2.0.0\",\n \"elm/json\": \"1.1.0\",\n \"elm/time\": \"1.0.0\",\n \"elm/url\": \"1.0.0\",\n \"elm-community/list-extra\": \"8.1.0\",\n \"jweir/elm-iso8601\": \"5.0.2\",\n \"rundis/elm-bootstrap\": \"5.0.0\"\n },\n \"indirect\": {\n \"avh4/elm-color\": \"1.0.0\",\n \"elm/bytes\": \"1.0.7\",\n \"elm/regex\": \"1.0.0\",\n \"elm/virtual-dom\": \"1.0.0\",\n \"myrho/elm-round\": \"1.0.4\"\n }\n },\n \"test-dependencies\": {\n \"direct\": {},\n \"indirect\": {}\n }\n}" }, { "path": "gateway/elm/src/Clipboard.elm", "content": "port module Clipboard exposing (copyToClipboard)\n\nimport Json.Encode as E\n\n\nport copyToClipboard : E.Value -> Cmd msg\n" }, { "path": "gateway/elm/src/Commands.elm", "content": "module Commands exposing\n ( Commit\n , Diff\n , DiffPair\n , Entry\n , Folder\n , HistoryEntry\n , ListResponse\n , Log\n , LoginResponse\n , Remote\n , SelfResponse\n , WhoamiResponse\n , diffChangeCount\n , doCopy\n , doDeletedFiles\n , doHistory\n , doListAllDirs\n , doListQuery\n , doLog\n , doLogin\n , doLogout\n , doMkdir\n , doMove\n , doPin\n , doRemoteAdd\n , doRemoteDiff\n , doRemoteList\n , doRemoteModify\n , doRemoteRemove\n , doRemoteSync\n , doRemove\n , doReset\n , doSelfQuery\n , doUndelete\n , doUnpin\n , doUpload\n , doWhoami\n , emptyRemote\n , emptySelf\n )\n\nimport Bootstrap.Dropdown as Dropdown\nimport File\nimport Http\nimport ISO8601\nimport Json.Decode as D\nimport Json.Decode.Pipeline as DP\nimport Json.Encode as E\nimport Time\nimport Url\nimport Util\n\n\n\n-- REMOVE\n\n\ntype alias RemoveQuery =\n { paths : List String\n }\n\n\nencodeRemoveQuery : RemoveQuery -> E.Value\nencodeRemoveQuery q =\n E.object\n [ ( \"paths\", E.list E.string q.paths ) ]\n\n\ndecodeRemoveResponse : D.Decoder String\ndecodeRemoveResponse =\n D.field \"message\" D.string\n\n\ndoRemove : (Result Http.Error String -> msg) -> List String -> Cmd msg\ndoRemove toMsg paths =\n Http.post\n { url = \"/api/v0/remove\"\n , body = Http.jsonBody <| encodeRemoveQuery <| RemoveQuery paths\n , expect = Http.expectJson toMsg decodeRemoveResponse\n }\n\n\n\n-- HISTORY\n\n\ntimestampToPosix : D.Decoder Time.Posix\ntimestampToPosix =\n D.int\n |> D.andThen\n (\\ms -> D.succeed <| Time.millisToPosix ms)\n\n\niso8601ToPosix : D.Decoder Time.Posix\niso8601ToPosix =\n D.string\n |> D.andThen\n (\\stamp ->\n case ISO8601.fromString stamp of\n Ok time ->\n D.succeed <| ISO8601.toPosix time\n\n Err msg ->\n D.fail msg\n )\n\n\ntype alias HistoryQuery =\n { path : String\n }\n\n\ntype alias Commit =\n { date : Time.Posix\n , msg : String\n , tags : List String\n , hash : String\n , index : Int\n }\n\n\ntype alias HistoryEntry =\n { head : Commit\n , path : String\n , change : String\n , isPinned : Bool\n , isExplicit : Bool\n }\n\n\nencodeHistoryQuery : HistoryQuery -> E.Value\nencodeHistoryQuery q =\n E.object\n [ ( \"path\", E.string q.path ) ]\n\n\ndecodeCommit : D.Decoder Commit\ndecodeCommit =\n D.succeed Commit\n |> DP.required \"date\" timestampToPosix\n |> DP.required \"msg\" D.string\n |> DP.required \"tags\" (D.list D.string)\n |> DP.required \"hash\" D.string\n |> DP.required \"index\" D.int\n\n\ndecodeHistoryEntry : D.Decoder HistoryEntry\ndecodeHistoryEntry =\n D.succeed HistoryEntry\n |> DP.required \"head\" decodeCommit\n |> DP.required \"path\" D.string\n |> DP.required \"change\" D.string\n |> DP.required \"is_pinned\" D.bool\n |> DP.required \"is_explicit\" D.bool\n\n\ndecodeHistory : D.Decoder (List HistoryEntry)\ndecodeHistory =\n D.field \"entries\" (D.list decodeHistoryEntry)\n\n\ndoHistory : (Result Http.Error (List HistoryEntry) -> msg) -> String -> Cmd msg\ndoHistory toMsg path =\n Http.post\n { url = \"/api/v0/history\"\n , body = Http.jsonBody <| encodeHistoryQuery <| HistoryQuery path\n , expect = Http.expectJson toMsg decodeHistory\n }\n\n\n\n-- RESET\n\n\ntype alias ResetQuery =\n { path : String\n , revision : String\n , force : Bool\n }\n\n\nencodeResetQuery : ResetQuery -> E.Value\nencodeResetQuery q =\n E.object\n [ ( \"path\", E.string q.path )\n , ( \"revision\", E.string q.revision )\n ]\n\n\ndecodeResetQuery : D.Decoder String\ndecodeResetQuery =\n D.field \"message\" D.string\n\n\ndoReset : (Result Http.Error String -> msg) -> String -> String -> Cmd msg\ndoReset toMsg path revision =\n Http.post\n { url = \"/api/v0/reset\"\n , body = Http.jsonBody <| encodeResetQuery <| ResetQuery path revision True\n , expect = Http.expectJson toMsg decodeResetQuery\n }\n\n\n\n-- MOVE\n\n\ntype alias MoveQuery =\n { sourcePath : String\n , destinationPath : String\n }\n\n\nencodeMoveQuery : MoveQuery -> E.Value\nencodeMoveQuery q =\n E.object\n [ ( \"source\", E.string <| Util.prefixSlash q.sourcePath )\n , ( \"destination\", E.string <| Util.prefixSlash q.destinationPath )\n ]\n\n\ndecodeMoveResponse : D.Decoder String\ndecodeMoveResponse =\n D.field \"message\" D.string\n\n\ndoMove : (Result Http.Error String -> msg) -> String -> String -> Cmd msg\ndoMove toMsg src dst =\n Http.post\n { url = \"/api/v0/move\"\n , body = Http.jsonBody <| encodeMoveQuery <| MoveQuery src dst\n , expect = Http.expectJson toMsg decodeMoveResponse\n }\n\n\n\n-- COPY\n\n\ntype alias CopyQuery =\n { sourcePath : String\n , destinationPath : String\n }\n\n\nencodeCopyQuery : CopyQuery -> E.Value\nencodeCopyQuery q =\n E.object\n [ ( \"source\", E.string <| Util.prefixSlash q.sourcePath )\n , ( \"destination\", E.string <| Util.prefixSlash q.destinationPath )\n ]\n\n\ndecodeCopyResponse : D.Decoder String\ndecodeCopyResponse =\n D.field \"message\" D.string\n\n\ndoCopy : (Result Http.Error String -> msg) -> String -> String -> Cmd msg\ndoCopy toMsg src dst =\n Http.post\n { url = \"/api/v0/copy\"\n , body = Http.jsonBody <| encodeCopyQuery <| CopyQuery src dst\n , expect = Http.expectJson toMsg decodeCopyResponse\n }\n\n\n\n-- ALL DIRS\n\n\ndecodeAllDirsResponse : D.Decoder (List String)\ndecodeAllDirsResponse =\n D.field \"paths\" (D.list D.string)\n\n\ndoListAllDirs : (Result Http.Error (List String) -> msg) -> Cmd msg\ndoListAllDirs toMsg =\n Http.post\n { url = \"/api/v0/all-dirs\"\n , body = Http.emptyBody\n , expect = Http.expectJson toMsg decodeAllDirsResponse\n }\n\n\n\n-- LIST\n\n\ntype alias ListQuery =\n { root : String\n , filter : String\n }\n\n\ntype alias Entry =\n { dropdown : Dropdown.State\n , path : String\n , user : String\n , size : Int\n , inode : Int\n , depth : Int\n , lastModified : Time.Posix\n , isDir : Bool\n , isPinned : Bool\n , isExplicit : Bool\n }\n\n\ntype alias ListResponse =\n { self : Entry\n , isFiltered : Bool\n , entries : List Entry\n }\n\n\nencodeListResponse : ListQuery -> E.Value\nencodeListResponse q =\n E.object\n [ ( \"root\", E.string q.root )\n , ( \"filter\", E.string q.filter )\n ]\n\n\ndecodeListResponse : D.Decoder ListResponse\ndecodeListResponse =\n D.map3 ListResponse\n (D.field \"self\" decodeEntry)\n (D.field \"is_filtered\" D.bool)\n (D.field \"files\" (D.list decodeEntry))\n\n\ndecodeEntry : D.Decoder Entry\ndecodeEntry =\n D.succeed (Entry Dropdown.initialState)\n |> DP.required \"path\" D.string\n |> DP.required \"user\" D.string\n |> DP.required \"size\" D.int\n |> DP.required \"inode\" D.int\n |> DP.required \"depth\" D.int\n |> DP.required \"last_modified_ms\" timestampToPosix\n |> DP.required \"is_dir\" D.bool\n |> DP.required \"is_pinned\" D.bool\n |> DP.required \"is_explicit\" D.bool\n\n\ndoListQuery : (Result Http.Error ListResponse -> msg) -> String -> String -> Cmd msg\ndoListQuery toMsg path filter =\n Http.post\n { url = \"/api/v0/ls\"\n , body = Http.jsonBody <| encodeListResponse <| ListQuery path filter\n , expect = Http.expectJson toMsg decodeListResponse\n }\n\n\n\n-- UPLOAD\n\n\ndoUpload : (String -> Result Http.Error () -> msg) -> String -> File.File -> Cmd msg\ndoUpload toMsg destPath file =\n Http.request\n { method = \"POST\"\n , url = \"/api/v0/upload?root=\" ++ Url.percentEncode destPath\n , headers = []\n , body = Http.multipartBody [ Http.filePart \"files[]\" file ]\n , expect = Http.expectWhatever (toMsg (File.name file))\n , timeout = Nothing\n , tracker = Just (\"upload-\" ++ File.name file)\n }\n\n\n\n-- MKDIR\n\n\ntype alias MkdirQuery =\n { path : String\n }\n\n\nencodeMkdirQuery : MkdirQuery -> E.Value\nencodeMkdirQuery q =\n E.object\n [ ( \"path\", E.string q.path ) ]\n\n\ndecodeMkdirResponse : D.Decoder String\ndecodeMkdirResponse =\n D.field \"message\" D.string\n\n\ndoMkdir : (Result Http.Error String -> msg) -> String -> Cmd msg\ndoMkdir toMsg path =\n Http.post\n { url = \"/api/v0/mkdir\"\n , body = Http.jsonBody <| encodeMkdirQuery <| MkdirQuery path\n , expect = Http.expectJson toMsg decodeMkdirResponse\n }\n\n\n\n-- LOGIN\n\n\ntype alias LoginQuery =\n { username : String\n , password : String\n }\n\n\ntype alias LoginResponse =\n { username : String\n , rights : List String\n , isAnon : Bool\n , anonIsAllowed : Bool\n }\n\n\nencodeLoginQuery : LoginQuery -> E.Value\nencodeLoginQuery q =\n E.object\n [ ( \"username\", E.string q.username )\n , ( \"password\", E.string q.password )\n ]\n\n\ndecodeLoginResponse : D.Decoder LoginResponse\ndecodeLoginResponse =\n D.map4 LoginResponse\n (D.field \"username\" D.string)\n (D.field \"rights\" (D.list D.string))\n (D.field \"is_anon\" D.bool)\n (D.field \"anon_is_allowed\" D.bool)\n\n\ndoLogin : (Result Http.Error LoginResponse -> msg) -> String -> String -> Cmd msg\ndoLogin toMsg user pass =\n Http.post\n { url = \"/api/v0/login\"\n , body = Http.jsonBody <| encodeLoginQuery <| LoginQuery user pass\n , expect = Http.expectJson toMsg decodeLoginResponse\n }\n\n\n\n-- LOGOUT QUERY\n\n\ndoLogout : (Result Http.Error Bool -> msg) -> Cmd msg\ndoLogout msg =\n Http.post\n { url = \"/api/v0/logout\"\n , body = Http.emptyBody\n , expect = Http.expectJson msg (D.field \"success\" D.bool)\n }\n\n\n\n-- WHOAMI QUERY\n\n\ntype alias WhoamiResponse =\n { username : String\n , isLoggedIn : Bool\n , isAnon : Bool\n , anonIsAllowed : Bool\n , rights : List String\n }\n\n\ndecodeWhoami : D.Decoder WhoamiResponse\ndecodeWhoami =\n D.map5 WhoamiResponse\n (D.field \"user\" D.string)\n (D.field \"is_logged_in\" D.bool)\n (D.field \"is_anon\" D.bool)\n (D.field \"anon_is_allowed\" D.bool)\n (D.field \"rights\" (D.list D.string))\n\n\ndoWhoami : (Result Http.Error WhoamiResponse -> msg) -> Cmd msg\ndoWhoami msg =\n Http.post\n { url = \"/api/v0/whoami\"\n , body = Http.emptyBody\n , expect = Http.expectJson msg decodeWhoami\n }\n\n\n\n-- LOG\n\n\ntype alias LogQuery =\n { offset : Int\n , limit : Int\n , filter : String\n }\n\n\ntype alias Log =\n { haveStagedChanges : Bool\n , commits : List Commit\n }\n\n\nencodeLog : LogQuery -> E.Value\nencodeLog q =\n E.object\n [ ( \"offset\", E.int q.offset )\n , ( \"limit\", E.int q.limit )\n , ( \"filter\", E.string q.filter )\n ]\n\n\ndecodeLog : D.Decoder Log\ndecodeLog =\n D.map2 Log\n (D.field \"have_staged_changes\" D.bool)\n (D.field \"commits\" (D.list decodeCommit))\n\n\ndoLog : (Result Http.Error Log -> msg) -> Int -> Int -> String -> Cmd msg\ndoLog msg offset limit filter =\n Http.post\n { url = \"/api/v0/log\"\n , body = Http.jsonBody <| encodeLog <| LogQuery offset limit filter\n , expect = Http.expectJson msg decodeLog\n }\n\n\n\n-- DELETED FILES\n\n\ntype alias DeletedFilesQuery =\n { offset : Int\n , limit : Int\n , filter : String\n }\n\n\nencodeDeletedFiles : DeletedFilesQuery -> E.Value\nencodeDeletedFiles q =\n E.object\n [ ( \"offset\", E.int q.offset )\n , ( \"limit\", E.int q.limit )\n , ( \"filter\", E.string q.filter )\n ]\n\n\ndecodeDeletedFiles : D.Decoder (List Entry)\ndecodeDeletedFiles =\n D.field \"entries\" (D.list decodeEntry)\n\n\ndoDeletedFiles : (Result Http.Error (List Entry) -> msg) -> Int -> Int -> String -> Cmd msg\ndoDeletedFiles msg offset limit filter =\n Http.post\n { url = \"/api/v0/deleted\"\n , body = Http.jsonBody <| encodeDeletedFiles <| DeletedFilesQuery offset limit filter\n , expect = Http.expectJson msg decodeDeletedFiles\n }\n\n\n\n-- UNDELETE\n\n\ntype alias UndeleteQuery =\n { path : String\n }\n\n\nencodeUndeleteQuery : UndeleteQuery -> E.Value\nencodeUndeleteQuery q =\n E.object\n [ ( \"path\", E.string q.path ) ]\n\n\ndecodeUndeleteResponse : D.Decoder String\ndecodeUndeleteResponse =\n D.field \"message\" D.string\n\n\ndoUndelete : (Result Http.Error String -> msg) -> String -> Cmd msg\ndoUndelete toMsg path =\n Http.post\n { url = \"/api/v0/undelete\"\n , body = Http.jsonBody <| encodeUndeleteQuery <| UndeleteQuery path\n , expect = Http.expectJson toMsg decodeUndeleteResponse\n }\n\n\n\n-- REMOTE LIST\n\n\ntype alias Folder =\n { folder : String\n , readOnly : Bool\n , conflictStrategy : String\n }\n\n\ntype alias Remote =\n { name : String\n , folders : List Folder\n , fingerprint : String\n , acceptAutoUpdates : Bool\n , isOnline : Bool\n , isAuthenticated : Bool\n , lastSeen : Time.Posix\n , acceptPush : Bool\n , conflictStrategy : String\n }\n\n\nemptyRemote : Remote\nemptyRemote =\n { name = \"\"\n , folders = []\n , fingerprint = \"\"\n , acceptAutoUpdates = False\n , isOnline = False\n , isAuthenticated = False\n , lastSeen = Time.millisToPosix 0\n , conflictStrategy = \"\"\n , acceptPush = False\n }\n\n\ndecodeRemoteListResponse : D.Decoder (List Remote)\ndecodeRemoteListResponse =\n D.field \"remotes\" (D.list decodeRemote)\n\n\ndecodeRemote : D.Decoder Remote\ndecodeRemote =\n D.succeed Remote\n |> DP.required \"name\" D.string\n |> DP.required \"folders\" (D.oneOf [ D.list decodeFolder, D.null [] ])\n |> DP.required \"fingerprint\" D.string\n |> DP.required \"accept_auto_updates\" D.bool\n |> DP.required \"is_online\" D.bool\n |> DP.required \"is_authenticated\" D.bool\n |> DP.required \"last_seen\" iso8601ToPosix\n |> DP.required \"accept_push\" D.bool\n |> DP.required \"conflict_strategy\" D.string\n\n\ndecodeFolder : D.Decoder Folder\ndecodeFolder =\n D.succeed Folder\n |> DP.required \"folder\" D.string\n |> DP.required \"read_only\" D.bool\n |> DP.required \"conflict_strategy\" D.string\n\n\ndoRemoteList : (Result Http.Error (List Remote) -> msg) -> Cmd msg\ndoRemoteList toMsg =\n Http.post\n { url = \"/api/v0/remotes/list\"\n , body = Http.emptyBody\n , expect = Http.expectJson toMsg decodeRemoteListResponse\n }\n\n\n\n-- REMOTE REMOVE\n\n\ntype alias RemoteRemoveQuery =\n { name : String\n }\n\n\nencodeRemoteRemoveQuery : RemoteRemoveQuery -> E.Value\nencodeRemoteRemoveQuery q =\n E.object\n [ ( \"name\", E.string q.name ) ]\n\n\ndecodeRemoteRemoveQuery : D.Decoder String\ndecodeRemoteRemoveQuery =\n D.field \"message\" D.string\n\n\ndoRemoteRemove : (Result Http.Error String -> msg) -> String -> Cmd msg\ndoRemoteRemove toMsg name =\n Http.post\n { url = \"/api/v0/remotes/remove\"\n , body = Http.jsonBody <| encodeRemoteRemoveQuery <| RemoteRemoveQuery name\n , expect = Http.expectJson toMsg decodeRemoteRemoveQuery\n }\n\n\n\n-- REMOTE SYNC\n\n\ntype alias RemoteSyncQuery =\n { name : String\n }\n\n\nencodeRemoteSyncQuery : RemoteSyncQuery -> E.Value\nencodeRemoteSyncQuery q =\n E.object\n [ ( \"name\", E.string q.name ) ]\n\n\ndecodeRemoteSyncQuery : D.Decoder String\ndecodeRemoteSyncQuery =\n D.field \"message\" D.string\n\n\ndoRemoteSync : (Result Http.Error String -> msg) -> String -> Cmd msg\ndoRemoteSync toMsg name =\n Http.post\n { url = \"/api/v0/remotes/sync\"\n , body = Http.jsonBody <| encodeRemoteSyncQuery <| RemoteSyncQuery name\n , expect = Http.expectJson toMsg decodeRemoteSyncQuery\n }\n\n\n\n-- REMOTE ADD\n\n\ntype alias RemoteAddQuery =\n { name : String\n , fingerprint : String\n , folders : List Folder\n , doAutoUpdate : Bool\n , acceptPush : Bool\n , conflictStrategy : String\n }\n\n\nencodeFolder : Folder -> E.Value\nencodeFolder f =\n E.object\n [ ( \"folder\", E.string f.folder )\n , ( \"read_only\", E.bool f.readOnly )\n , ( \"conflict_strategy\", E.string f.conflictStrategy )\n ]\n\n\nencodeRemoteAddQuery : RemoteAddQuery -> E.Value\nencodeRemoteAddQuery q =\n E.object\n [ ( \"name\", E.string q.name )\n , ( \"fingerprint\", E.string q.fingerprint )\n , ( \"accept_auto_updates\", E.bool q.doAutoUpdate )\n , ( \"folders\", E.list encodeFolder q.folders )\n , ( \"accept_push\", E.bool q.acceptPush )\n , ( \"conflict_strategy\", E.string q.conflictStrategy )\n ]\n\n\ndecodeRemoteAddQuery : D.Decoder String\ndecodeRemoteAddQuery =\n D.field \"message\" D.string\n\n\ndoRemoteAdd : (Result Http.Error String -> msg) -> String -> String -> Bool -> Bool -> String -> List Folder -> Cmd msg\ndoRemoteAdd toMsg name fingerprint doAutoUpdate acceptPush conflictStrategy folders =\n Http.post\n { url = \"/api/v0/remotes/add\"\n , body =\n Http.jsonBody <|\n encodeRemoteAddQuery <|\n { name = name\n , fingerprint = fingerprint\n , doAutoUpdate = doAutoUpdate\n , folders = folders\n , acceptPush = acceptPush\n , conflictStrategy = conflictStrategy\n }\n , expect = Http.expectJson toMsg decodeRemoteAddQuery\n }\n\n\ndoRemoteModify : (Result Http.Error String -> msg) -> Remote -> Cmd msg\ndoRemoteModify toMsg remote =\n Http.post\n { url = \"/api/v0/remotes/modify\"\n , body =\n Http.jsonBody <|\n encodeRemoteAddQuery <|\n { name = remote.name\n , fingerprint = remote.fingerprint\n , doAutoUpdate = remote.acceptAutoUpdates\n , folders = remote.folders\n , acceptPush = remote.acceptPush\n , conflictStrategy = remote.conflictStrategy\n }\n , expect = Http.expectJson toMsg decodeRemoteAddQuery\n }\n\n\n\n-- REMOTE SELF\n\n\ntype alias SelfResponse =\n { self : Identity\n , defaultConflictStrategy : String\n }\n\n\ntype alias Identity =\n { name : String\n , fingerprint : String\n }\n\n\nemptySelf : SelfResponse\nemptySelf =\n SelfResponse (Identity \"\" \"\") \"marker\"\n\n\ndecodeSelfResponse : D.Decoder SelfResponse\ndecodeSelfResponse =\n D.succeed SelfResponse\n |> DP.required \"self\" decodeIdentity\n |> DP.required \"default_conflict_strategy\" D.string\n\n\ndecodeIdentity : D.Decoder Identity\ndecodeIdentity =\n D.succeed Identity\n |> DP.required \"name\" D.string\n |> DP.required \"fingerprint\" D.string\n\n\ndoSelfQuery : (Result Http.Error SelfResponse -> msg) -> Cmd msg\ndoSelfQuery toMsg =\n Http.post\n { url = \"/api/v0/remotes/self\"\n , body = Http.emptyBody\n , expect = Http.expectJson toMsg decodeSelfResponse\n }\n\n\n\n-- REMOTE DIFF\n\n\ntype alias DiffPair =\n { src : Entry\n , dst : Entry\n }\n\n\ntype alias Diff =\n { added : List Entry\n , removed : List Entry\n , ignored : List Entry\n , missing : List Entry\n , moved : List DiffPair\n , merged : List DiffPair\n , conflict : List DiffPair\n }\n\n\ndiffChangeCount : Diff -> Int\ndiffChangeCount diff =\n List.length diff.added\n + List.length diff.removed\n + List.length diff.ignored\n + List.length diff.missing\n + List.length diff.moved\n + List.length diff.merged\n + List.length diff.conflict\n\n\ntype alias RemoteDiffQuery =\n { name : String\n }\n\n\nencodeRemoteDiffQuery : RemoteDiffQuery -> E.Value\nencodeRemoteDiffQuery q =\n E.object\n [ ( \"name\", E.string q.name ) ]\n\n\ndecodeDiffPair : D.Decoder DiffPair\ndecodeDiffPair =\n D.map2 DiffPair\n (D.field \"src\" decodeEntry)\n (D.field \"dst\" decodeEntry)\n\n\ndecodeDiffResponse : D.Decoder Diff\ndecodeDiffResponse =\n D.field \"diff\" decodeDiff\n\n\ndecodeDiff : D.Decoder Diff\ndecodeDiff =\n D.succeed Diff\n |> DP.required \"added\" (D.list decodeEntry)\n |> DP.required \"removed\" (D.list decodeEntry)\n |> DP.required \"ignored\" (D.list decodeEntry)\n |> DP.required \"missing\" (D.list decodeEntry)\n |> DP.required \"moved\" (D.list decodeDiffPair)\n |> DP.required \"merged\" (D.list decodeDiffPair)\n |> DP.required \"conflict\" (D.list decodeDiffPair)\n\n\ndoRemoteDiff : (Result Http.Error Diff -> msg) -> String -> Cmd msg\ndoRemoteDiff toMsg name =\n Http.post\n { url = \"/api/v0/remotes/diff\"\n , body = Http.jsonBody <| encodeRemoteDiffQuery <| RemoteDiffQuery name\n , expect = Http.expectJson toMsg decodeDiffResponse\n }\n\n\n\n-- PIN\n\n\ntype alias PinQuery =\n { path : String\n , revision : String\n }\n\n\nencodePinQuery : PinQuery -> E.Value\nencodePinQuery q =\n E.object\n [ ( \"path\", E.string q.path )\n , ( \"revision\", E.string q.revision )\n ]\n\n\ndecodePinResponse : D.Decoder String\ndecodePinResponse =\n D.field \"message\" D.string\n\n\ndoPin : (Result Http.Error String -> msg) -> String -> String -> Cmd msg\ndoPin toMsg path revision =\n Http.post\n { url = \"/api/v0/pin\"\n , body = Http.jsonBody <| encodePinQuery <| PinQuery path revision\n , expect = Http.expectJson toMsg decodePinResponse\n }\n\n\ndoUnpin : (Result Http.Error String -> msg) -> String -> String -> Cmd msg\ndoUnpin toMsg path revision =\n Http.post\n { url = \"/api/v0/unpin\"\n , body = Http.jsonBody <| encodePinQuery <| PinQuery path revision\n , expect = Http.expectJson toMsg decodePinResponse\n }\n" }, { "path": "gateway/elm/src/Main.elm", "content": "module Main exposing (init, main, subscriptions, update, view)\n\nimport Bootstrap.Alert as Alert\nimport Bootstrap.Button as Button\nimport Bootstrap.Form as Form\nimport Bootstrap.Form.Input as Input\nimport Bootstrap.Grid as Grid\nimport Bootstrap.Grid.Col as Col\nimport Bootstrap.Text as Text\nimport Browser\nimport Browser.Navigation as Nav\nimport Commands\nimport Html exposing (..)\nimport Html.Attributes exposing (..)\nimport Html.Events exposing (..)\nimport Html.Lazy as Lazy\nimport Http\nimport Json.Decode as D\nimport List\nimport Pinger\nimport Routes.Commits as Commits\nimport Routes.DeletedFiles as DeletedFiles\nimport Routes.Diff as Diff\nimport Routes.Ls as Ls\nimport Routes.Remotes as Remotes\nimport Task\nimport Time\nimport Url\nimport Util exposing (..)\nimport Websocket\n\n\n\n-- MAIN\n\n\nmain : Program () Model Msg\nmain =\n Browser.application\n { init = init\n , update = update\n , subscriptions = subscriptions\n , view = view\n , onUrlChange = UrlChanged\n , onUrlRequest = LinkClicked\n }\n\n\n\n-- MESSAGES\n\n\ntype Msg\n = GotLoginResp (Result Http.Error Commands.LoginResponse)\n | GotWhoamiResp (Result Http.Error Commands.WhoamiResponse)\n | GotLogoutResp Bool (Result Http.Error Bool)\n | AdjustTimeZone Time.Zone\n | LinkClicked Browser.UrlRequest\n | UrlChanged Url.Url\n | UsernameInput String\n | PasswordInput String\n | LoginSubmit\n | LogoutSubmit Bool\n | GotoLogin\n | PingerIn String\n | WebsocketIn String\n -- View parent messages:\n | ListMsg Ls.Msg\n | CommitsMsg Commits.Msg\n | DeletedFilesMsg DeletedFiles.Msg\n | RemotesMsg Remotes.Msg\n | DiffMsg Diff.Msg\n\n\n\n-- MODEL\n\n\ntype View\n = ViewList\n | ViewCommits\n | ViewRemotes\n | ViewDeletedFiles\n | ViewDiff\n | ViewNotFound\n\n\ntype alias ViewState =\n { listState : Ls.Model\n , commitsState : Commits.Model\n , remoteState : Remotes.Model\n , deletedFilesState : DeletedFiles.Model\n , diffState : Diff.Model\n , loginName : String\n , currentView : View\n , rights : List String\n , isAnon : Bool\n , anonIsAllowed : Bool\n }\n\n\ntype LoginState\n = LoginLimbo -- weird state where we do not know if we're logged in yet.\n | LoginReady String String\n | LoginLoading String String\n | LoginFailure String String String\n | LoginSuccess ViewState\n\n\ntype alias Model =\n { zone : Time.Zone\n , key : Nav.Key\n , url : Url.Url\n , loginState : LoginState\n , serverIsOnline : Bool\n }\n\n\ninit : () -> Url.Url -> Nav.Key -> ( Model, Cmd Msg )\ninit _ url key =\n ( { zone = Time.utc\n , key = key\n , url = url\n , loginState = LoginLimbo\n , serverIsOnline = True\n }\n , Cmd.batch\n [ Task.perform AdjustTimeZone Time.here\n , Commands.doWhoami GotWhoamiResp\n ]\n )\n\n\n\n-- UPDATE\n\n\nwithSubUpdate : subMsg -> (ViewState -> subModel) -> Model -> (subMsg -> Msg) -> (subMsg -> subModel -> ( subModel, Cmd subMsg )) -> (ViewState -> subModel -> ViewState) -> ( Model, Cmd Msg )\nwithSubUpdate subMsg subModel model msg subUpdate viewStateUpdate =\n case model.loginState of\n LoginSuccess viewState ->\n let\n ( newSubModel, newSubCmd ) =\n subUpdate subMsg (subModel viewState)\n in\n ( { model | loginState = LoginSuccess (viewStateUpdate viewState newSubModel) }, Cmd.map msg newSubCmd )\n\n _ ->\n ( model, Cmd.none )\n\n\ndoInitAfterLogin : Model -> String -> List String -> Bool -> Bool -> ( Model, Cmd Msg )\ndoInitAfterLogin model loginName rights isAnon anonIsAllowed =\n let\n newViewState =\n { listState = Ls.newModel model.key model.url rights\n , commitsState = Commits.newModel model.url model.key model.zone rights\n , deletedFilesState = DeletedFiles.newModel model.url model.key model.zone rights\n , remoteState = Remotes.newModel model.key model.zone rights\n , diffState = Diff.newModel model.key model.url model.zone\n , loginName = loginName\n , currentView = viewFromUrl rights model.url\n , rights = rights\n , isAnon = isAnon\n , anonIsAllowed = anonIsAllowed\n }\n in\n ( { model | loginState = LoginSuccess newViewState }\n , Cmd.batch\n [ Cmd.map ListMsg <| Ls.doListQueryFromUrl model.url\n , Websocket.open ()\n , Cmd.map DeletedFilesMsg <| DeletedFiles.reload newViewState.deletedFilesState\n , Cmd.map CommitsMsg <| Commits.reload newViewState.commitsState\n , Cmd.map RemotesMsg <| Remotes.reload\n , Cmd.map DiffMsg <| Diff.reload newViewState.diffState model.url\n ]\n )\n\n\nviewFromUrl : List String -> Url.Url -> View\nviewFromUrl rights url =\n case List.head <| List.drop 1 <| String.split \"/\" url.path of\n Nothing ->\n ViewNotFound\n\n Just first ->\n case first of\n \"view\" ->\n ViewList\n\n \"log\" ->\n ViewCommits\n\n \"remotes\" ->\n ViewRemotes\n\n \"deleted\" ->\n ViewDeletedFiles\n\n \"diff\" ->\n ViewDiff\n\n \"\" ->\n if List.member \"fs.view\" rights then\n ViewList\n\n else\n ViewRemotes\n\n _ ->\n ViewNotFound\n\n\nviewToString : View -> String\nviewToString v =\n case v of\n ViewList ->\n \"/view\"\n\n ViewCommits ->\n \"/log\"\n\n ViewRemotes ->\n \"/remotes\"\n\n ViewDeletedFiles ->\n \"/deleted\"\n\n ViewDiff ->\n \"/Diff\"\n\n ViewNotFound ->\n \"/nothing\"\n\n\neventType : String -> String\neventType data =\n let\n result =\n D.decodeString (D.field \"data\" D.string) data\n in\n case result of\n Ok typ ->\n typ\n\n Err _ ->\n \"failed\"\n\n\npingerMsgToBool : String -> Bool\npingerMsgToBool data =\n let\n result =\n D.decodeString (D.field \"isOnline\" D.bool) data\n in\n case result of\n Ok typ ->\n typ\n\n Err _ ->\n False\n\n\nupdate : Msg -> Model -> ( Model, Cmd Msg )\nupdate msg model =\n case msg of\n AdjustTimeZone newZone ->\n case model.loginState of\n LoginSuccess viewState ->\n ( { model\n | zone = newZone\n , loginState = LoginSuccess { viewState | listState = Ls.changeTimeZone newZone viewState.listState }\n }\n , Cmd.none\n )\n\n _ ->\n ( { model | zone = newZone }, Cmd.none )\n\n GotWhoamiResp result ->\n case result of\n Ok whoami ->\n -- Immediately hit off a list query, which will in turn populate\n -- the list view. Take the path from the current URL.\n case whoami.isLoggedIn of\n True ->\n doInitAfterLogin model whoami.username whoami.rights whoami.isAnon whoami.anonIsAllowed\n\n False ->\n ( { model | loginState = LoginReady \"\" \"\" }, Cmd.none )\n\n Err _ ->\n ( { model | loginState = LoginReady \"\" \"\" }, Cmd.none )\n\n GotLoginResp result ->\n case result of\n Ok response ->\n -- Immediately hit off a list query, which will in turn populate\n -- the list view. Take the path from the current URL.\n doInitAfterLogin model response.username response.rights response.isAnon response.anonIsAllowed\n\n Err err ->\n ( { model | loginState = LoginFailure \"\" \"\" (Util.httpErrorToString err) }, Cmd.none )\n\n GotLogoutResp mayReloginAsAnon _ ->\n case model.loginState of\n LoginSuccess viewState ->\n if mayReloginAsAnon && viewState.anonIsAllowed then\n ( model, Commands.doWhoami GotWhoamiResp )\n\n else\n ( { model | loginState = LoginReady \"\" \"\" }, Cmd.none )\n\n _ ->\n ( { model | loginState = LoginReady \"\" \"\" }, Cmd.none )\n\n LinkClicked urlRequest ->\n case urlRequest of\n Browser.Internal url ->\n -- Special case: /get/ requests should be routed as if we clicked an\n -- external link.\n case String.startsWith \"/get\" url.path of\n True ->\n ( model, Nav.load (Url.toString url) )\n\n False ->\n ( model, Nav.pushUrl model.key (Url.toString { url | query = Nothing }) )\n\n Browser.External href ->\n let\n currUrl =\n Url.toString model.url\n in\n case href of\n \"\" ->\n ( model, Cmd.none )\n\n \"#\" ->\n ( model, Cmd.none )\n\n _ ->\n ( model\n , if href == currUrl then\n Cmd.none\n\n else\n Nav.load href\n )\n\n UrlChanged url ->\n case model.loginState of\n LoginSuccess viewState ->\n case viewFromUrl viewState.rights url of\n ViewList ->\n ( { model\n | url = url\n , loginState =\n LoginSuccess\n { viewState\n | currentView = ViewList\n , listState = Ls.changeUrl url viewState.listState\n , commitsState = Commits.updateUrl viewState.commitsState url\n , deletedFilesState = DeletedFiles.updateUrl viewState.deletedFilesState url\n , diffState = Diff.updateUrl viewState.diffState url\n }\n }\n , Cmd.map ListMsg <| Ls.doListQueryFromUrl url\n )\n\n ViewDiff ->\n ( { model\n | url = url\n , loginState =\n LoginSuccess\n { viewState\n | currentView = ViewDiff\n , commitsState = Commits.updateUrl viewState.commitsState url\n , deletedFilesState = DeletedFiles.updateUrl viewState.deletedFilesState url\n , diffState = Diff.updateUrl viewState.diffState url\n }\n }\n , Cmd.map DiffMsg <| Diff.reload viewState.diffState url\n )\n\n ViewCommits ->\n ( { model\n | url = url\n , loginState =\n LoginSuccess\n { viewState\n | currentView = ViewCommits\n , commitsState = Commits.updateUrl viewState.commitsState url\n , deletedFilesState = DeletedFiles.updateUrl viewState.deletedFilesState url\n , diffState = Diff.updateUrl viewState.diffState url\n }\n }\n , Cmd.map CommitsMsg <| Commits.reloadIfNeeded viewState.commitsState\n )\n\n ViewDeletedFiles ->\n ( { model\n | url = url\n , loginState =\n LoginSuccess\n { viewState\n | currentView = ViewDeletedFiles\n , deletedFilesState = DeletedFiles.updateUrl viewState.deletedFilesState url\n , commitsState = Commits.updateUrl viewState.commitsState url\n , diffState = Diff.updateUrl viewState.diffState url\n }\n }\n , Cmd.map DeletedFilesMsg <| DeletedFiles.reloadIfNeeded viewState.deletedFilesState\n )\n\n other ->\n ( { model\n | url = url\n , loginState =\n LoginSuccess\n { viewState\n | currentView = other\n , commitsState = Commits.updateUrl viewState.commitsState url\n , deletedFilesState = DeletedFiles.updateUrl viewState.deletedFilesState url\n , diffState = Diff.updateUrl viewState.diffState url\n }\n }\n , Cmd.none\n )\n\n _ ->\n ( { model | url = url }, Cmd.none )\n\n UsernameInput username ->\n case model.loginState of\n LoginReady _ password ->\n ( { model | loginState = LoginReady username password }, Cmd.none )\n\n LoginFailure _ password _ ->\n ( { model | loginState = LoginFailure username password \"\" }, Cmd.none )\n\n _ ->\n ( model, Cmd.none )\n\n PasswordInput password ->\n case model.loginState of\n LoginReady username _ ->\n ( { model | loginState = LoginReady username password }, Cmd.none )\n\n LoginFailure username _ _ ->\n ( { model | loginState = LoginFailure username password \"\" }, Cmd.none )\n\n _ ->\n ( model, Cmd.none )\n\n LoginSubmit ->\n case model.loginState of\n LoginReady username password ->\n ( { model | loginState = LoginLoading username password }\n , Commands.doLogin GotLoginResp username password\n )\n\n LoginFailure username password _ ->\n ( { model | loginState = LoginLoading username password }\n , Commands.doLogin GotLoginResp username password\n )\n\n _ ->\n ( model, Cmd.none )\n\n LogoutSubmit mayReloginAsAnon ->\n ( model, Commands.doLogout (GotLogoutResp mayReloginAsAnon) )\n\n GotoLogin ->\n ( { model | loginState = LoginReady \"\" \"\" }, Cmd.none )\n\n PingerIn pingMsg ->\n ( { model | serverIsOnline = pingerMsgToBool pingMsg }, Cmd.none )\n\n WebsocketIn event ->\n -- The backend lets us know that some of the data changed.\n -- Depending on the event type these are currently either\n -- filesystem entries or remotes.\n case eventType event of\n \"pin\" ->\n case model.loginState of\n LoginSuccess viewState ->\n ( model, Cmd.map ListMsg <| Ls.doListQueryFromUrl model.url )\n\n _ ->\n ( model, Cmd.none )\n\n \"fs\" ->\n case model.loginState of\n LoginSuccess viewState ->\n ( model\n , Cmd.batch\n [ Cmd.map ListMsg <| Ls.doListQueryFromUrl model.url\n , Cmd.map DeletedFilesMsg <| DeletedFiles.reload viewState.deletedFilesState\n , Cmd.map CommitsMsg <| Commits.reload viewState.commitsState\n ]\n )\n\n _ ->\n ( model, Cmd.none )\n\n \"remotes\" ->\n ( model, Cmd.map RemotesMsg Remotes.reload )\n\n _ ->\n ( model, Cmd.none )\n\n ListMsg subMsg ->\n withSubUpdate\n subMsg\n .listState\n model\n ListMsg\n Ls.update\n (\\viewState newSubModel -> { viewState | listState = newSubModel })\n\n CommitsMsg subMsg ->\n withSubUpdate\n subMsg\n .commitsState\n model\n CommitsMsg\n Commits.update\n (\\viewState newSubModel -> { viewState | commitsState = newSubModel })\n\n DeletedFilesMsg subMsg ->\n withSubUpdate\n subMsg\n .deletedFilesState\n model\n DeletedFilesMsg\n DeletedFiles.update\n (\\viewState newSubModel -> { viewState | deletedFilesState = newSubModel })\n\n RemotesMsg subMsg ->\n withSubUpdate\n subMsg\n .remoteState\n model\n RemotesMsg\n Remotes.update\n (\\viewState newSubModel -> { viewState | remoteState = newSubModel })\n\n DiffMsg subMsg ->\n withSubUpdate\n subMsg\n .diffState\n model\n DiffMsg\n Diff.update\n (\\viewState newSubModel -> { viewState | diffState = newSubModel })\n\n\n\n-- VIEW\n\n\nview : Model -> Browser.Document Msg\nview model =\n { title = \"Gateway\"\n , body =\n case model.loginState of\n LoginLimbo ->\n [ text \"Waiting for login data\" ]\n\n LoginReady _ _ ->\n [ Lazy.lazy viewLoginForm model ]\n\n LoginFailure _ _ _ ->\n [ Lazy.lazy viewLoginForm model ]\n\n LoginLoading _ _ ->\n [ Lazy.lazy viewLoginForm model ]\n\n LoginSuccess viewState ->\n viewMainContent model viewState\n }\n\n\nviewAppIcon : Model -> Html Msg\nviewAppIcon model =\n a [ class \"nav-link active\", href \"/view\" ]\n (if model.serverIsOnline then\n [ span [ class \"fas fa-2x fa-fw logo fa-torii-gate\" ] []\n , span [ class \"badge badge-success text-center\" ] [ text \"beta\" ]\n ]\n\n else\n [ span [ class \"fas fa-2x fa-fw logo logo-failure fa-torii-gate\" ] []\n , span [ class \"badge badge-danger text-center\" ] [ text \"offline\" ]\n ]\n )\n\n\nviewOfflineMarker : Html Msg\nviewOfflineMarker =\n div [ class \"row h-100\" ]\n [ div\n [ class \"col-12 my-auto text-center w-100 text-muted\" ]\n [ span [ class \"fas fa-4x fa-fw logo-failure fa-plug\" ] []\n , br [] []\n , br [] []\n , text \"It seems that we have lost connection to the server.\"\n , br [] []\n , text \"This application will go into a working state again when we have a connection again.\"\n ]\n ]\n\n\nviewMainContent : Model -> ViewState -> List (Html Msg)\nviewMainContent model viewState =\n [ div [ class \"container-fluid\" ]\n [ div [ class \"row wrapper\" ]\n [ aside [ class \"col-12 col-md-2 p-0 bg-light tabbar\" ]\n [ nav [ class \"navbar navbar-expand-md navbar-light bg-align-items-start flex-md-column flex-row\" ]\n [ viewAppIcon model\n , a [ class \"navbar-toggler\", attribute \"data-toggle\" \"collapse\", attribute \"data-target\" \".sidebar\" ]\n [ span [ class \"navbar-toggler-icon\" ] []\n ]\n , div [ class \"collapse navbar-collapse sidebar\" ]\n [ viewSidebarItems model viewState\n ]\n ]\n , viewSidebarBottom model\n ]\n , main_ [ class \"col\" ]\n (if model.serverIsOnline then\n [ viewCurrentRoute model viewState\n , Html.map ListMsg (Ls.buildModals viewState.listState)\n , Html.map RemotesMsg (Remotes.buildModals viewState.remoteState)\n ]\n\n else\n [ viewOfflineMarker ]\n )\n ]\n ]\n ]\n\n\nviewCurrentRoute : Model -> ViewState -> Html Msg\nviewCurrentRoute model viewState =\n case viewState.currentView of\n ViewList ->\n Html.map ListMsg <| Ls.view viewState.listState\n\n ViewCommits ->\n Html.map CommitsMsg <| Commits.view viewState.commitsState\n\n ViewDeletedFiles ->\n Html.map DeletedFilesMsg <| DeletedFiles.view viewState.deletedFilesState\n\n ViewRemotes ->\n Html.map RemotesMsg <| Remotes.view viewState.remoteState\n\n ViewDiff ->\n Html.map DiffMsg <| Diff.view viewState.diffState\n\n ViewNotFound ->\n text \"You seem to have hit a route that does not exist...\"\n\n\nviewLoginInputs : String -> String -> List (Html Msg)\nviewLoginInputs username password =\n [ h2 [ class \"login-header\" ] [ text \"Login\" ]\n , Input.text\n [ Input.id \"username-input\"\n , Input.attrs [ class \"login-input\" ]\n , Input.large\n , Input.placeholder \"Username\"\n , Input.value username\n , Input.onInput UsernameInput\n ]\n , Input.password\n [ Input.id \"password-input\"\n , Input.attrs [ class \"login-input\" ]\n , Input.large\n , Input.placeholder \"Password\"\n , Input.value password\n , Input.onInput PasswordInput\n ]\n ]\n\n\nviewLoginButton : String -> String -> Bool -> Html Msg\nviewLoginButton username password isLoading =\n let\n loadingClass =\n if isLoading then\n \"fa fa-sync fa-sync-animate\"\n\n else\n \"\"\n in\n Button.button\n [ Button.primary\n , Button.attrs\n [ onClick <| LoginSubmit\n , class \"login-btn\"\n , type_ \"submit\"\n , disabled\n (String.length (String.trim username)\n == 0\n || String.length (String.trim password)\n == 0\n || isLoading\n )\n ]\n ]\n [ span [ class loadingClass ] [], text \" Log in\" ]\n\n\nviewLoginForm : Model -> Html Msg\nviewLoginForm model =\n Grid.containerFluid \n [ class \"login-background\" ]\n [ Grid.row []\n [ Grid.col\n [ Col.lg8\n , Col.textAlign Text.alignXsCenter\n , Col.attrs [ class \"login-form\" ]\n ]\n [ Form.form [ onSubmit LoginSubmit ]\n [ Form.group []\n (case model.loginState of\n LoginReady username password ->\n viewLoginInputs username password\n ++ [ viewLoginButton username password False ]\n\n LoginLoading username password ->\n viewLoginInputs username password\n ++ [ viewLoginButton username password True ]\n\n LoginFailure username password _ ->\n viewLoginInputs username password\n ++ [ Alert.simpleDanger [] [ text \"Login failed, please try again.\" ]\n , viewLoginButton username password False\n ]\n\n _ ->\n -- This should not happen.\n []\n )\n ]\n ]\n ]\n ]\n\n\nhasRight : ViewState -> String -> List (Html Msg) -> List (Html Msg)\nhasRight viewState right elements =\n if List.member right viewState.rights then\n elements\n\n else\n []\n\n\nviewSidebarItems : Model -> ViewState -> Html Msg\nviewSidebarItems model viewState =\n let\n isActiveClass =\n \\v ->\n if v == viewState.currentView then\n class \"nav-link active\"\n\n else\n class \"nav-link\"\n in\n ul [ class \"flex-column navbar-nav w-100 text-left\" ]\n (hasRight viewState\n \"fs.view\"\n [ li [ class \"nav-item\" ]\n [ a [ isActiveClass ViewList, href (viewToString ViewList) ]\n [ span [] [ text \"Files\" ] ]\n ]\n ]\n ++ hasRight viewState\n \"fs.view\"\n [ li [ class \"nav-item\" ]\n [ a [ isActiveClass ViewCommits, href (viewToString ViewCommits) ]\n [ span [] [ text \"Changelog\" ] ]\n ]\n ]\n ++ hasRight viewState\n \"fs.view\"\n [ li [ class \"nav-item\" ]\n [ a [ isActiveClass ViewDeletedFiles, href (viewToString ViewDeletedFiles) ]\n [ span [] [ text \"Trashbin\" ] ]\n ]\n ]\n ++ hasRight viewState\n \"remotes.view\"\n [ li [ class \"nav-item\" ]\n [ a [ isActiveClass ViewRemotes, href (viewToString ViewRemotes) ]\n [ span [] [ text \"Remotes\" ] ]\n ]\n ]\n ++ (if viewState.isAnon then\n [ li [ class \"nav-item\" ]\n [ a [ class \"nav-link pl-0\", href \"#\", onClick (LogoutSubmit False) ]\n [ span [] [ text \"Login page\" ] ]\n ]\n ]\n\n else\n [ li [ class \"nav-item\" ]\n [ a [ class \"nav-link pl-0\", href \"#\", onClick (LogoutSubmit True) ]\n [ span [] [ text (\"Logout »\" ++ viewState.loginName ++ \"«\") ] ]\n ]\n ]\n )\n )\n\n\nviewSidebarBottom : Model -> Html Msg\nviewSidebarBottom model =\n -- Make sure to not display that on small devices:\n div [ id \"sidebar-bottom\", class \"d-none d-lg-block\" ]\n [ hr [] []\n , p [ id \"sidebar-bottom-text\", class \"text-muted\" ]\n [ span []\n [ text \"Powered by \"\n , a [ href \"https://github.com/sahib/brig\" ] [ text \"brig\" ]\n ]\n ]\n ]\n\n\n\n-- SUBSCRIPTIONS\n\n\nsubscriptions : Model -> Sub Msg\nsubscriptions model =\n case model.loginState of\n LoginSuccess viewState ->\n Sub.batch\n [ Sub.map ListMsg (Ls.subscriptions viewState.listState)\n , Sub.map CommitsMsg (Commits.subscriptions viewState.commitsState)\n , Sub.map RemotesMsg (Remotes.subscriptions viewState.remoteState)\n , Sub.map DeletedFilesMsg (DeletedFiles.subscriptions viewState.deletedFilesState)\n , Websocket.incoming WebsocketIn\n , Pinger.pinger PingerIn\n ]\n\n _ ->\n Sub.none\n" }, { "path": "gateway/elm/src/Modals/History.elm", "content": "module Modals.History exposing (Model, Msg, newModel, show, subscriptions, update, view)\n\nimport Bootstrap.Alert as Alert\nimport Bootstrap.Button as Button\nimport Bootstrap.ButtonGroup as ButtonGroup\nimport Bootstrap.Grid as Grid\nimport Bootstrap.Grid.Col as Col\nimport Bootstrap.Grid.Row as Row\nimport Bootstrap.ListGroup as ListGroup\nimport Bootstrap.Modal as Modal\nimport Browser.Events as Events\nimport Commands\nimport Html exposing (..)\nimport Html.Attributes exposing (..)\nimport Html.Events exposing (..)\nimport Http\nimport Json.Decode as D\nimport List\nimport Time\nimport Util\n\n\ntype alias Model =\n { modal : Modal.Visibility\n , alert : Alert.Visibility\n , history : Maybe (Result Http.Error (List Commands.HistoryEntry))\n , rights : List String\n , lastPath : String\n }\n\n\ntype Msg\n = ModalShow\n | GotHistoryResponse String (Result Http.Error (List Commands.HistoryEntry))\n | GotResetResponse (Result Http.Error String)\n | GotPinResponse (Result Http.Error String)\n | ResetClicked String String\n | AnimateModal Modal.Visibility\n | AlertMsg Alert.Visibility\n | ModalClose\n | KeyPress String\n | PinClicked String String Bool\n\n\n\n-- INIT\n\n\nnewModel : List String -> Model\nnewModel rights =\n { modal = Modal.hidden\n , alert = Alert.shown\n , history = Nothing\n , rights = rights\n , lastPath = \"\"\n }\n\n\n\n-- UPDATE\n\n\nupdate : Msg -> Model -> ( Model, Cmd Msg )\nupdate msg model =\n case msg of\n GotHistoryResponse path result ->\n ( { model\n | modal = Modal.shown\n , history = Just result\n , lastPath = path\n }\n , Cmd.none\n )\n\n ResetClicked path revision ->\n ( model, Commands.doReset GotResetResponse path revision )\n\n GotResetResponse result ->\n case result of\n Ok _ ->\n ( { model | modal = Modal.hidden, history = Nothing }, Cmd.none )\n\n Err err ->\n ( { model | history = Just (Err err) }, Cmd.none )\n\n GotPinResponse result ->\n case result of\n Ok _ ->\n -- Update the whole history to get the latest pin state.\n -- This is kinda wasteful and might be optimized later if we need to.\n ( model, Commands.doHistory (GotHistoryResponse model.lastPath) model.lastPath )\n\n Err err ->\n ( { model | history = Just (Err err) }, Cmd.none )\n\n AnimateModal visibility ->\n ( { model | modal = visibility }, Cmd.none )\n\n ModalShow ->\n ( { model | modal = Modal.shown }, Cmd.none )\n\n ModalClose ->\n ( { model | modal = Modal.hidden, history = Nothing }, Cmd.none )\n\n AlertMsg vis ->\n ( { model | alert = vis }, Cmd.none )\n\n PinClicked path revision shouldBePinned ->\n ( model\n , if shouldBePinned then\n Commands.doPin GotPinResponse path revision\n\n else\n Commands.doUnpin GotPinResponse path revision\n )\n\n KeyPress key ->\n if model.modal == Modal.hidden then\n ( model, Cmd.none )\n\n else\n case key of\n \"Enter\" ->\n ( { model | modal = Modal.hidden, history = Nothing }, Cmd.none )\n\n _ ->\n ( model, Cmd.none )\n\n\n\n-- VIEW\n\n\nviewChangeColor : String -> Html Msg\nviewChangeColor change =\n case change of\n \"added\" ->\n span [ class \"text-success\" ] [ text change ]\n\n \"modified\" ->\n span [ class \"text-warning\" ] [ text change ]\n\n \"removed\" ->\n span [ class \"text-danger\" ] [ text change ]\n\n \"moved\" ->\n span [ class \"text-info\" ] [ text change ]\n\n _ ->\n span [ class \"text-muted\" ] [ text change ]\n\n\njoinChanges : List (Html Msg) -> List (Html Msg)\njoinChanges changes =\n List.intersperse (text \", \") changes\n\n\nviewChangeSet : String -> Html Msg\nviewChangeSet change =\n let\n changes =\n List.map viewChangeColor (String.split \"|\" change)\n in\n span [] (joinChanges changes)\n\n\nviewPinIcon : Bool -> Bool -> Html msg\nviewPinIcon isPinned isExplicit =\n case ( isPinned, isExplicit ) of\n ( True, True ) ->\n span\n [ class \"fa fa-map-marker\", class \"text-success\" ]\n []\n\n ( True, False ) ->\n span\n [ class \"fa fa-map-marker-alt\", class \"text-warning\" ]\n []\n\n _ ->\n span\n [ class \"fa fa-times\", class \"text-danger\" ]\n []\n\n\nviewPinButton : Model -> Commands.HistoryEntry -> Html Msg\nviewPinButton model entry =\n Button.button\n [ Button.outlinePrimary\n , Button.attrs\n [ disabled (not (List.member \"fs.edit\" model.rights))\n , onClick (PinClicked entry.path entry.head.hash (not entry.isPinned))\n ]\n ]\n [ viewPinIcon entry.isPinned entry.isExplicit ]\n\n\nviewHistoryEntry : Model -> Bool -> Commands.HistoryEntry -> Html Msg\nviewHistoryEntry model isFirst entry =\n Grid.row []\n [ Grid.col [ Col.xs9 ]\n [ p []\n [ text entry.path\n , br [] []\n , viewChangeSet entry.change\n , span [ class \"text-muted\" ] [ text \" at \" ]\n , text <| Util.formatLastModified Time.utc entry.head.date\n , text \": \"\n , span [ class \"text-muted\" ] [ text entry.head.msg ]\n ]\n ]\n , Grid.col [ Col.xs3 ]\n [ ButtonGroup.buttonGroup\n []\n [ ButtonGroup.button\n [ Button.outlinePrimary\n , Button.attrs\n [ onClick <| ResetClicked entry.path entry.head.hash\n , disabled isFirst\n ]\n ]\n [ text \"Revert\" ]\n , ButtonGroup.button\n [ Button.outlinePrimary\n , Button.attrs\n [ disabled (not (List.member \"fs.edit\" model.rights))\n , onClick (PinClicked entry.path entry.head.hash (not entry.isPinned))\n ]\n ]\n [ viewPinIcon entry.isPinned entry.isExplicit ]\n ]\n ]\n ]\n\n\nviewHistoryEntries : Model -> List Commands.HistoryEntry -> Html Msg\nviewHistoryEntries model entries =\n Grid.row []\n [ Grid.col []\n [ ListGroup.ul\n (List.indexedMap (\\idx e -> ListGroup.li [] [ viewHistoryEntry model (idx == 0) e ]) entries)\n ]\n ]\n\n\nviewHistory : Model -> List (Grid.Column Msg)\nviewHistory model =\n [ Grid.col [ Col.xs12 ]\n [ case model.history of\n Nothing ->\n text \"\"\n\n Just result ->\n case result of\n Ok entries ->\n viewHistoryEntries model entries\n\n Err err ->\n Util.buildAlert\n model.alert\n AlertMsg\n Alert.danger\n \"Oh no!\"\n (\"Could not read history: \" ++ Util.httpErrorToString err)\n ]\n ]\n\n\nview : Model -> Html Msg\nview model =\n Modal.config ModalClose\n |> Modal.large\n |> Modal.withAnimation AnimateModal\n |> Modal.header [ class \"modal-title modal-header-success\" ]\n [ h4 [] [ text \"History\" ] ]\n |> Modal.body []\n [ Grid.containerFluid []\n [ Grid.row [ Row.attrs [ class \"scrollable-modal-row\" ] ] (viewHistory model) ]\n ]\n |> Modal.footer []\n [ Button.button\n [ Button.outlinePrimary\n , Button.attrs [ onClick <| AnimateModal Modal.hiddenAnimated ]\n ]\n [ text \"Close\" ]\n ]\n |> Modal.view model.modal\n\n\nshow : String -> Cmd Msg\nshow path =\n Commands.doHistory (GotHistoryResponse path) path\n\n\n\n-- SUBSCRIPTIONS\n\n\nsubscriptions : Model -> Sub Msg\nsubscriptions model =\n Sub.batch\n [ Modal.subscriptions model.modal AnimateModal\n , Alert.subscriptions model.alert AlertMsg\n , Events.onKeyPress (D.map KeyPress <| D.field \"key\" D.string)\n ]\n" }, { "path": "gateway/elm/src/Modals/Mkdir.elm", "content": "module Modals.Mkdir exposing (Model, Msg, newModel, show, subscriptions, update, view)\n\nimport Bootstrap.Alert as Alert\nimport Bootstrap.Button as Button\nimport Bootstrap.Form.Input as Input\nimport Bootstrap.Grid as Grid\nimport Bootstrap.Grid.Col as Col\nimport Bootstrap.Modal as Modal\nimport Browser.Events as Events\nimport Commands\nimport Html exposing (..)\nimport Html.Attributes exposing (..)\nimport Html.Events exposing (..)\nimport Http\nimport Json.Decode as D\nimport Url\nimport Util\n\n\ntype State\n = Ready\n | Fail String\n\n\ntype alias Model =\n { state : State\n , inputName : String\n , modal : Modal.Visibility\n , alert : Alert.Visibility\n }\n\n\ntype Msg\n = CreateDir String\n | InputChanged String\n | ModalShow\n | GotResponse (Result Http.Error String)\n | AnimateModal Modal.Visibility\n | AlertMsg Alert.Visibility\n | ModalClose\n | KeyPress String String\n\n\n\n-- INIT\n\n\nnewModel : Model\nnewModel =\n { state = Ready\n , modal = Modal.hidden\n , inputName = \"\"\n , alert = Alert.shown\n }\n\n\n\n-- UPDATE\n\n\nupdate : Msg -> Model -> ( Model, Cmd Msg )\nupdate msg model =\n case msg of\n CreateDir path ->\n ( model, Commands.doMkdir GotResponse path )\n\n InputChanged inputName ->\n ( { model | inputName = inputName }, Cmd.none )\n\n GotResponse result ->\n case result of\n Ok _ ->\n -- New list model means also new checked entries.\n ( { model | state = Ready, modal = Modal.hidden }, Cmd.none )\n\n Err err ->\n ( { model | state = Fail <| Util.httpErrorToString err }, Cmd.none )\n\n AnimateModal visibility ->\n ( { model | modal = visibility }, Cmd.none )\n\n ModalShow ->\n ( { model | modal = Modal.shown, inputName = \"\" }, Cmd.none )\n\n ModalClose ->\n ( { model | modal = Modal.hidden, state = Ready }, Cmd.none )\n\n AlertMsg vis ->\n ( { model | alert = vis }, Cmd.none )\n\n KeyPress path key ->\n if model.modal == Modal.hidden then\n ( model, Cmd.none )\n\n else\n case key of\n \"Enter\" ->\n ( model, Commands.doMkdir GotResponse path )\n\n _ ->\n ( model, Cmd.none )\n\n\n\n-- VIEW\n\n\nshowPathCollision : Model -> Bool -> Html Msg\nshowPathCollision model doesExist =\n if doesExist then\n span [ class \"text-left\" ]\n [ span [ class \"fas fa-md fa-exclamation-triangle text-warning\" ] []\n , span [ class \"text-muted\" ]\n [ text (\" »\" ++ model.inputName ++ \"« exists already. Please choose another name.\\u{00A0}\\u{00A0}\\u{00A0}\")\n ]\n ]\n\n else\n span [] []\n\n\nviewMkdirContent : Model -> List (Grid.Column Msg)\nviewMkdirContent model =\n [ Grid.col [ Col.xs12 ]\n [ Input.text\n [ Input.id \"mkdir-input\"\n , Input.large\n , Input.placeholder \"Directory name\"\n , Input.onInput InputChanged\n , Input.attrs [ autofocus True ]\n ]\n , br [] []\n , case model.state of\n Ready ->\n text \"\"\n\n Fail message ->\n Util.buildAlert model.alert AlertMsg Alert.danger \"Oh no!\" (\"Could not create directory: \" ++ message)\n ]\n ]\n\n\npathFromUrl : Url.Url -> Model -> String\npathFromUrl url model =\n Util.joinPath [ Util.urlToPath url, model.inputName ]\n\n\nview : Model -> Url.Url -> (String -> Bool) -> Html Msg\nview model url existChecker =\n let\n path =\n Util.urlToPath url\n\n hasPathCollision =\n existChecker model.inputName\n in\n Modal.config ModalClose\n |> Modal.large\n |> Modal.withAnimation AnimateModal\n |> Modal.header [ class \"modal-title modal-header-primary\" ]\n [ h4 []\n [ text \"Create a new directory in \"\n , span []\n [ text \"»\"\n , text\n (if path == \"/\" then\n \"Home\"\n\n else\n path\n )\n , text \"«\"\n ]\n ]\n ]\n |> Modal.body []\n [ Grid.containerFluid []\n [ Grid.row [] (viewMkdirContent model) ]\n ]\n |> Modal.footer []\n [ showPathCollision model hasPathCollision\n , Button.button\n [ Button.primary\n , Button.attrs\n [ onClick (CreateDir (pathFromUrl url model))\n , type_ \"submit\"\n , disabled\n (String.length model.inputName\n == 0\n || (case model.state of\n Fail _ ->\n True\n\n _ ->\n False\n )\n || hasPathCollision\n )\n ]\n ]\n [ text \"Create\" ]\n , Button.button\n [ Button.outlinePrimary\n , Button.attrs [ onClick <| AnimateModal Modal.hiddenAnimated ]\n ]\n [ text \"Cancel\" ]\n ]\n |> Modal.view model.modal\n\n\nshow : Msg\nshow =\n ModalShow\n\n\n\n-- SUBSCRIPTIONS\n\n\nsubscriptions : Url.Url -> Model -> Sub Msg\nsubscriptions url model =\n Sub.batch\n [ Modal.subscriptions model.modal AnimateModal\n , Alert.subscriptions model.alert AlertMsg\n , Events.onKeyPress (D.map (KeyPress <| pathFromUrl url model) <| D.field \"key\" D.string)\n ]\n" }, { "path": "gateway/elm/src/Modals/MoveCopy.elm", "content": "module Modals.MoveCopy exposing\n ( Model\n , Msg\n , newCopyModel\n , newMoveModel\n , show\n , subscriptions\n , update\n , view\n , viewDirList\n , viewSearchBox\n )\n\nimport Bootstrap.Alert as Alert\nimport Bootstrap.Button as Button\nimport Bootstrap.Form.Input as Input\nimport Bootstrap.Form.InputGroup as InputGroup\nimport Bootstrap.Grid as Grid\nimport Bootstrap.Grid.Col as Col\nimport Bootstrap.Grid.Row as Row\nimport Bootstrap.Modal as Modal\nimport Bootstrap.Table as Table\nimport Browser.Events as Events\nimport Commands\nimport Html exposing (..)\nimport Html.Attributes exposing (..)\nimport Html.Events exposing (..)\nimport Http\nimport Json.Decode as D\nimport List\nimport Util\n\n\ntype State\n = Ready (List String)\n | Loading\n | Fail String\n\n\ntype alias Model =\n { state : State\n , action : Type\n , destPath : String\n , sourcePath : String\n , filter : String\n , modal : Modal.Visibility\n , alert : Alert.Visibility\n }\n\n\ntype Msg\n = DoAction\n | DirChosen String\n | SearchInput String\n | ModalShow String\n | GotAllDirsResponse (Result Http.Error (List String))\n | GotActionResponse (Result Http.Error String)\n | AnimateModal Modal.Visibility\n | AlertMsg Alert.Visibility\n | ModalClose\n | KeyPress String\n\n\n\n-- INIT\n\n\ntype Type\n = Move\n | Copy\n\n\ntypeToString : Type -> String\ntypeToString typ =\n case typ of\n Move ->\n \"Move\"\n\n Copy ->\n \"Copy\"\n\n\nnewMoveModel : Model\nnewMoveModel =\n { state = Loading\n , modal = Modal.hidden\n , action = Move\n , destPath = \"\"\n , sourcePath = \"\"\n , filter = \"\"\n , alert = Alert.shown\n }\n\n\nnewCopyModel : Model\nnewCopyModel =\n { state = Loading\n , modal = Modal.hidden\n , action = Copy\n , destPath = \"\"\n , sourcePath = \"\"\n , filter = \"\"\n , alert = Alert.shown\n }\n\n\n\n-- UPDATE\n\n\nfixPath : String -> String\nfixPath path =\n if path == \"/\" then\n \"Home\"\n\n else\n String.join \"/\" (Util.splitPath path)\n\n\nfilterInvalidTargets : String -> String -> Bool\nfilterInvalidTargets sourcePath path =\n (path /= Util.dirname sourcePath)\n && not (String.startsWith path sourcePath)\n\n\nfixAllDirResponse : Model -> List String -> List String\nfixAllDirResponse model paths =\n List.filter (filterInvalidTargets model.sourcePath) paths\n |> List.map fixPath\n\n\nfilterAllDirs : String -> List String -> List String\nfilterAllDirs filter dirs =\n let\n lowerFilter =\n String.toLower filter\n in\n List.filter (String.contains lowerFilter) dirs\n\n\ndoAction : Model -> Cmd Msg\ndoAction model =\n case model.action of\n Move ->\n Commands.doMove GotActionResponse model.sourcePath model.destPath\n\n Copy ->\n Commands.doCopy GotActionResponse model.sourcePath model.destPath\n\n\nupdate : Msg -> Model -> ( Model, Cmd Msg )\nupdate msg model =\n case msg of\n DoAction ->\n ( model, doAction model )\n\n DirChosen path ->\n ( { model | destPath = path }, Cmd.none )\n\n SearchInput filter ->\n ( { model | filter = filter }, Cmd.none )\n\n GotAllDirsResponse result ->\n case result of\n Ok dirs ->\n ( { model | state = Ready (fixAllDirResponse model dirs) }, Cmd.none )\n\n Err err ->\n ( { model | state = Fail <| Util.httpErrorToString err }, Cmd.none )\n\n GotActionResponse result ->\n case result of\n Ok _ ->\n ( { model | modal = Modal.hidden }, Cmd.none )\n\n Err err ->\n ( { model | state = Fail <| Util.httpErrorToString err }, Cmd.none )\n\n AnimateModal visibility ->\n ( { model | modal = visibility }, Cmd.none )\n\n ModalShow sourcePath ->\n ( { model\n | modal = Modal.shown\n , sourcePath = sourcePath\n , destPath = \"\"\n , state = Loading\n }\n , Commands.doListAllDirs GotAllDirsResponse\n )\n\n ModalClose ->\n ( { model | modal = Modal.hidden }, Cmd.none )\n\n AlertMsg vis ->\n ( { model | alert = vis }, Cmd.none )\n\n KeyPress key ->\n ( model\n , if model.modal == Modal.hidden || model.destPath == \"\" then\n Cmd.none\n\n else\n case key of\n \"Enter\" ->\n doAction model\n\n _ ->\n Cmd.none\n )\n\n\n\n-- VIEW\n\n\nviewDirEntry : (String -> msg) -> String -> Table.Row msg\nviewDirEntry clickMsg path =\n Table.tr []\n [ Table.td\n [ Table.cellAttr <| onClick (clickMsg path) ]\n [ span [ class \"fas fa-lg fa-folder text-xs-right file-list-icon\" ] [] ]\n , Table.td\n [ Table.cellAttr <| onClick (clickMsg path) ]\n [ text path ]\n ]\n\n\nviewDirList : (String -> msg) -> String -> List String -> Html msg\nviewDirList clickMsg filter dirs =\n Table.table\n { options = [ Table.hover ]\n , thead =\n Table.thead [ Table.headAttr (style \"display\" \"none\") ]\n [ Table.tr []\n [ Table.th [ Table.cellAttr (style \"width\" \"10%\") ] []\n , Table.th [ Table.cellAttr (style \"width\" \"90%\") ] []\n ]\n ]\n , tbody =\n Table.tbody [] (List.map (viewDirEntry clickMsg) (filterAllDirs filter dirs))\n }\n\n\nviewSearchBox : (String -> msg) -> String -> Html msg\nviewSearchBox searchMsg filter =\n InputGroup.config\n (InputGroup.text\n [ Input.placeholder \"Filter directory list\"\n , Input.attrs\n [ onInput searchMsg\n , value filter\n ]\n ]\n )\n |> InputGroup.successors\n [ InputGroup.span [ class \"input-group-addon\" ]\n [ button [] [ span [ class \"fas fa-search fa-xs input-group-addon\" ] [] ]\n ]\n ]\n |> InputGroup.attrs [ class \"stylish-input-group input-group\" ]\n |> InputGroup.view\n\n\nviewContent : Model -> List (Grid.Column Msg)\nviewContent model =\n [ Grid.col [ Col.xs12 ]\n [ case model.state of\n Ready dirs ->\n div []\n [ viewSearchBox SearchInput model.filter\n , viewDirList DirChosen model.filter dirs\n ]\n\n Loading ->\n text \"Loading.\"\n\n Fail message ->\n Util.buildAlert\n model.alert\n AlertMsg\n Alert.danger\n \"Oh no!\"\n (\"Could not move or copy path: \" ++ message)\n ]\n ]\n\n\nview : Model -> Html Msg\nview model =\n Modal.config ModalClose\n |> Modal.large\n |> Modal.withAnimation AnimateModal\n |> Modal.header [ class \"modal-title modal-header-primary\" ]\n [ h4 []\n [ text (typeToString model.action ++ \" \")\n , span []\n [ text \"»\"\n , text (Util.basename model.sourcePath)\n , text \"«\"\n ]\n , if String.length model.destPath > 0 then\n span []\n [ text \" into »\"\n , text model.destPath\n , text \"«\"\n ]\n\n else\n text \" into ...\"\n ]\n ]\n |> Modal.body []\n [ Grid.containerFluid []\n [ Grid.row\n [ Row.attrs [ class \"scrollable-modal-row\" ] ]\n (viewContent model)\n ]\n ]\n |> Modal.footer []\n [ Button.button\n [ Button.primary\n , Button.attrs\n [ onClick DoAction\n , type_ \"submit\"\n , disabled\n (String.length model.destPath\n == 0\n || (case model.state of\n Fail _ ->\n True\n\n _ ->\n False\n )\n )\n ]\n ]\n [ text (typeToString model.action) ]\n , Button.button\n [ Button.outlinePrimary\n , Button.attrs [ onClick <| AnimateModal Modal.hiddenAnimated ]\n ]\n [ text \"Cancel\" ]\n ]\n |> Modal.view model.modal\n\n\nshow : String -> Msg\nshow sourcePath =\n ModalShow sourcePath\n\n\n\n-- SUBSCRIPTIONS\n\n\nsubscriptions : Model -> Sub Msg\nsubscriptions model =\n Sub.batch\n [ Modal.subscriptions model.modal AnimateModal\n , Alert.subscriptions model.alert AlertMsg\n , Events.onKeyPress (D.map KeyPress <| D.field \"key\" D.string)\n ]\n" }, { "path": "gateway/elm/src/Modals/RemoteAdd.elm", "content": "module Modals.RemoteAdd exposing (Model, Msg, newModel, show, subscriptions, update, view)\n\nimport Bootstrap.Alert as Alert\nimport Bootstrap.Button as Button\nimport Bootstrap.Dropdown as Dropdown\nimport Bootstrap.Form.Input as Input\nimport Bootstrap.Grid as Grid\nimport Bootstrap.Grid.Col as Col\nimport Bootstrap.Modal as Modal\nimport Browser.Events as Events\nimport Commands\nimport Dict\nimport Html exposing (..)\nimport Html.Attributes exposing (..)\nimport Html.Events exposing (..)\nimport Http\nimport Json.Decode as D\nimport Util\n\n\ntype State\n = Ready\n | Fail String\n\n\ntype alias Model =\n { state : State\n , name : String\n , fingerprint : String\n , doAutoUdate : Bool\n , acceptPush : Bool\n , conflictStrategy : String\n , modal : Modal.Visibility\n , alert : Alert.Visibility\n , conflictDropdown : Dropdown.State\n }\n\n\ntype Msg\n = RemoteAdd\n | NameInputChanged String\n | FingerprintInputChanged String\n | AutoUpdateChanged Bool\n | AcceptPushChanged Bool\n | ConflictStrategyChanged String\n | ConflictDropdownMsg Dropdown.State\n | ModalShow\n | GotResponse (Result Http.Error String)\n | AnimateModal Modal.Visibility\n | AlertMsg Alert.Visibility\n | ModalClose\n | KeyPress String\n\n\n\n-- INIT\n\n\nnewModel : Model\nnewModel =\n newModelWithState Modal.hidden\n\n\nnewModelWithState : Modal.Visibility -> Model\nnewModelWithState state =\n { state = Ready\n , modal = state\n , name = \"\"\n , fingerprint = \"\"\n , doAutoUdate = False\n , acceptPush = False\n , alert = Alert.shown\n , conflictDropdown = Dropdown.initialState\n , conflictStrategy = \"\"\n }\n\n\n\n-- UPDATE\n\n\nsubmit : Model -> Cmd Msg\nsubmit model =\n Commands.doRemoteAdd GotResponse\n model.name\n model.fingerprint\n model.doAutoUdate\n model.acceptPush\n model.conflictStrategy\n []\n\n\nupdate : Msg -> Model -> ( Model, Cmd Msg )\nupdate msg model =\n case msg of\n RemoteAdd ->\n ( model, submit model )\n\n NameInputChanged name ->\n ( { model | name = name }, Cmd.none )\n\n FingerprintInputChanged fingerprint ->\n ( { model | fingerprint = fingerprint }, Cmd.none )\n\n AutoUpdateChanged doAutoUdate ->\n ( { model | doAutoUdate = doAutoUdate }, Cmd.none )\n\n AcceptPushChanged acceptPush ->\n ( { model | acceptPush = acceptPush }, Cmd.none )\n\n GotResponse result ->\n case result of\n Ok _ ->\n -- New list model means also new checked entries.\n ( { model | state = Ready, modal = Modal.hidden }, Cmd.none )\n\n Err err ->\n ( { model | state = Fail <| Util.httpErrorToString err }, Cmd.none )\n\n AnimateModal visibility ->\n ( { model | modal = visibility }, Cmd.none )\n\n ModalShow ->\n ( newModelWithState Modal.shown, Cmd.none )\n\n ModalClose ->\n ( { model | modal = Modal.hidden }, Cmd.none )\n\n AlertMsg vis ->\n ( { model | alert = vis }, Cmd.none )\n\n ConflictDropdownMsg state ->\n ( { model | conflictDropdown = state }, Cmd.none )\n\n ConflictStrategyChanged state ->\n ( { model | conflictStrategy = state }, Cmd.none )\n\n KeyPress key ->\n if model.modal == Modal.hidden then\n ( model, Cmd.none )\n\n else\n case key of\n \"Enter\" ->\n ( model, submit model )\n\n _ ->\n ( model, Cmd.none )\n\n\n\n-- VIEW\n\n\nviewRemoteAddContent : Model -> List (Grid.Column Msg)\nviewRemoteAddContent model =\n [ Grid.col [ Col.xs12 ]\n [ Input.text\n [ Input.id \"remote-name-input\"\n , Input.large\n , Input.placeholder \"Remote name\"\n , Input.onInput NameInputChanged\n , Input.attrs [ autofocus True ]\n ]\n , br [] []\n , Input.text\n [ Input.id \"remote-fingerprint-input\"\n , Input.large\n , Input.placeholder \"Remote fingerprint\"\n , Input.onInput FingerprintInputChanged\n ]\n , br [] []\n , span []\n [ Util.viewToggleSwitch\n AutoUpdateChanged\n \"Accept automatic updates?\"\n model.doAutoUdate\n False\n ]\n , br [] []\n , span []\n [ Util.viewToggleSwitch\n AcceptPushChanged\n \"Accept other remotes pushing data to us?\"\n model.acceptPush\n False\n ]\n , br [] []\n , span []\n [ span [ class \"text-muted\" ] [ text \"The current conflict strategy is\" ]\n , viewConflictDropdown model\n , span [ class \"text-muted\" ] [ text \".\" ]\n ]\n , case model.state of\n Ready ->\n text \"\"\n\n Fail message ->\n Util.buildAlert model.alert AlertMsg Alert.danger \"Oh no!\" (\"Could not add remote: \" ++ message)\n ]\n ]\n\n\nshowCurrentConflictStrategy : Model -> Html Msg\nshowCurrentConflictStrategy model =\n case model.conflictStrategy of\n \"\" ->\n span [] [ text \"Marker \", span [ class \"fas fa-marker\" ] [] ]\n\n \"ignore\" ->\n span [] [ text \"Ignore \", span [ class \"fas fa-eject\" ] [] ]\n\n \"marker\" ->\n span [] [ text \"Marker \", span [ class \"fas fa-marker\" ] [] ]\n\n \"embrace\" ->\n span [] [ text \"Embrace \", span [ class \"fas fa-handshake\" ] [] ]\n\n _ ->\n span [] [ text \"Unknown \", span [ class \"fas fa-question\" ] [] ]\n\n\nviewConflictDropdown : Model -> Html Msg\nviewConflictDropdown model =\n Dropdown.dropdown\n model.conflictDropdown\n { options =\n [ Dropdown.alignMenuRight\n , Dropdown.attrs [ id \"remote-add-conflict-dropdown\" ]\n ]\n , toggleMsg = ConflictDropdownMsg\n , toggleButton =\n Dropdown.toggle\n [ Button.roleLink ]\n [ showCurrentConflictStrategy model ]\n , items =\n [ Dropdown.buttonItem\n [ onClick (ConflictStrategyChanged \"ignore\") ]\n [ span [ class \"fas fa-md fa-eject\" ] [], text \" Ignore\" ]\n , Dropdown.buttonItem\n [ onClick (ConflictStrategyChanged \"marker\") ]\n [ span [ class \"fas fa-md fa-marker\" ] [], text \" Marker\" ]\n , Dropdown.buttonItem\n [ onClick (ConflictStrategyChanged \"embrace\") ]\n [ span [ class \"fas fa-md fa-handshake\" ] [], text \" Embrace\" ]\n , Dropdown.buttonItem\n [ onClick (ConflictStrategyChanged \"\") ]\n [ span [ class \"fas fa-md fa-eraser\" ] [], text \" Default\" ]\n ]\n }\n\n\nview : Model -> Html Msg\nview model =\n Modal.config ModalClose\n |> Modal.large\n |> Modal.withAnimation AnimateModal\n |> Modal.header [ class \"modal-title modal-header-primary\" ]\n [ h4 [] [ text \"Add a new remote\" ] ]\n |> Modal.body []\n [ Grid.containerFluid []\n [ Grid.row [] (viewRemoteAddContent model) ]\n ]\n |> Modal.footer []\n [ Button.button\n [ Button.primary\n , Button.attrs\n [ onClick RemoteAdd\n , type_ \"submit\"\n , disabled\n (String.length model.name\n == 0\n || String.length model.fingerprint\n == 0\n || (case model.state of\n Fail _ ->\n True\n\n _ ->\n False\n )\n )\n ]\n ]\n [ text \"Create\" ]\n , Button.button\n [ Button.outlinePrimary\n , Button.attrs [ onClick <| AnimateModal Modal.hiddenAnimated ]\n ]\n [ text \"Cancel\" ]\n ]\n |> Modal.view model.modal\n\n\nshow : Msg\nshow =\n ModalShow\n\n\n\n-- SUBSCRIPTIONS\n\n\nsubscriptions : Model -> Sub Msg\nsubscriptions model =\n Sub.batch\n [ Modal.subscriptions model.modal AnimateModal\n , Alert.subscriptions model.alert AlertMsg\n , Events.onKeyPress (D.map KeyPress <| D.field \"key\" D.string)\n , Dropdown.subscriptions model.conflictDropdown ConflictDropdownMsg\n ]\n" }, { "path": "gateway/elm/src/Modals/RemoteFolders.elm", "content": "module Modals.RemoteFolders exposing\n ( Model\n , Msg\n , newModel\n , show\n , subscriptions\n , update\n , view\n )\n\nimport Bootstrap.Alert as Alert\nimport Bootstrap.Button as Button\nimport Bootstrap.Dropdown as Dropdown\nimport Bootstrap.Form.Input as Input\nimport Bootstrap.Form.InputGroup as InputGroup\nimport Bootstrap.Grid as Grid\nimport Bootstrap.Grid.Col as Col\nimport Bootstrap.Grid.Row as Row\nimport Bootstrap.ListGroup as ListGroup\nimport Bootstrap.Modal as Modal\nimport Bootstrap.Table as Table\nimport Bootstrap.Text as Text\nimport Browser.Events as Events\nimport Commands\nimport Dict\nimport Html exposing (..)\nimport Html.Attributes exposing (..)\nimport Html.Events exposing (..)\nimport Http\nimport Json.Decode as D\nimport List.Extra as LE\nimport Modals.MoveCopy as MoveCopy\nimport Set\nimport Util\n\n\ntype State\n = Ready\n | Fail String\n\n\ntype alias Model =\n { state : State\n , allDirs : List String\n , filter : String\n , remote : Commands.Remote\n , modal : Modal.Visibility\n , alert : Alert.Visibility\n , conflictDropdowns : Dict.Dict String Dropdown.State\n }\n\n\ntype Msg\n = ModalShow Commands.Remote\n | FolderRemove String\n | ReadOnlyChanged String Bool\n | GotResponse (Result Http.Error String)\n | AnimateModal Modal.Visibility\n | AlertMsg Alert.Visibility\n | ModalClose\n | GotAllDirsResponse (Result Http.Error (List String))\n | DirChosen String\n | SearchInput String\n | ConflictStrategyToggled String String\n | ConflictDropdownMsg String Dropdown.State\n\n\n\n-- INIT\n\n\nnewModel : Model\nnewModel =\n newModelWithState Modal.hidden Commands.emptyRemote\n\n\nnewModelWithState : Modal.Visibility -> Commands.Remote -> Model\nnewModelWithState state remote =\n { state = Ready\n , modal = state\n , allDirs = []\n , filter = \"\"\n , remote = remote\n , alert = Alert.shown\n , conflictDropdowns = Dict.empty\n }\n\n\n\n-- UPDATE\n\n\nsubmit : Commands.Remote -> Cmd Msg\nsubmit remote =\n Commands.doRemoteModify GotResponse remote\n\n\nfixFolder : String -> String\nfixFolder path =\n Util.prefixSlash path\n\n\naddFolder : Model -> String -> ( Model, Cmd Msg )\naddFolder model folder =\n let\n oldRemote =\n model.remote\n\n cleanFolder =\n Commands.Folder (fixFolder folder) False \"\"\n\n newRemote =\n { oldRemote | folders = List.sortBy .folder <| cleanFolder :: oldRemote.folders }\n\n upModel =\n { model | remote = newRemote }\n in\n ( upModel, submit upModel.remote )\n\n\nupdate : Msg -> Model -> ( Model, Cmd Msg )\nupdate msg model =\n case msg of\n GotResponse result ->\n case result of\n Ok _ ->\n -- New list model means also new checked entries.\n ( { model | state = Ready }, Cmd.none )\n\n Err err ->\n ( { model | state = Fail <| Util.httpErrorToString err }, Cmd.none )\n\n FolderRemove folder ->\n let\n oldRemote =\n model.remote\n\n newRemote =\n { oldRemote | folders = List.filter (\\f -> f.folder /= folder) oldRemote.folders }\n\n upModel =\n { model | remote = newRemote }\n in\n ( upModel, submit upModel.remote )\n\n AnimateModal visibility ->\n ( { model | modal = visibility }, Cmd.none )\n\n ModalShow remote ->\n ( newModelWithState Modal.shown remote\n , Commands.doListAllDirs GotAllDirsResponse\n )\n\n GotAllDirsResponse result ->\n case result of\n Ok allDirs ->\n ( { model | allDirs = allDirs }, Cmd.none )\n\n Err _ ->\n ( model, Cmd.none )\n\n DirChosen choice ->\n addFolder model choice\n\n SearchInput filter ->\n ( { model | filter = filter }, Cmd.none )\n\n ModalClose ->\n ( { model | modal = Modal.hidden, filter = \"\" }, Cmd.none )\n\n AlertMsg vis ->\n ( { model | alert = vis }, Cmd.none )\n\n ReadOnlyChanged path state ->\n let\n oldRemote =\n model.remote\n\n newRemote =\n { oldRemote\n | folders =\n List.map\n (\\f ->\n if f.folder == path then\n { f | readOnly = state }\n\n else\n f\n )\n model.remote.folders\n }\n in\n ( { model | remote = newRemote }, submit newRemote )\n\n ConflictDropdownMsg folder state ->\n ( { model | conflictDropdowns = Dict.insert folder state model.conflictDropdowns }, Cmd.none )\n\n ConflictStrategyToggled folder strategy ->\n let\n oldRemote =\n model.remote\n\n newFolders =\n List.map\n (\\f ->\n if f.folder == folder then\n { f | conflictStrategy = strategy }\n\n else\n f\n )\n model.remote.folders\n\n newRemote =\n { oldRemote | folders = newFolders }\n\n upModel =\n { model | remote = newRemote }\n in\n ( upModel, submit upModel.remote )\n\n\n\n-- VIEW\n\n\nviewRow : Html Msg -> Html Msg -> Html Msg -> Html Msg\nviewRow a b c =\n Grid.row []\n [ Grid.col [ Col.xs1, Col.textAlign Text.alignXsRight ] [ a ]\n , Grid.col [ Col.xs8, Col.textAlign Text.alignXsLeft ] [ b ]\n , Grid.col [ Col.xs3, Col.textAlign Text.alignXsLeft ] [ c ]\n ]\n\n\nconflictStrategyToIconName : String -> String\nconflictStrategyToIconName strategy =\n case strategy of\n \"\" ->\n \"fa-marker text-muted\"\n\n \"ignore\" ->\n \"fa-eject\"\n\n \"marker\" ->\n \"fa-marker\"\n\n \"embrace\" ->\n \"fa-handshake\"\n\n _ ->\n \"fa-question\"\n\n\nviewConflictDropdown : Model -> Commands.Folder -> Html Msg\nviewConflictDropdown model folder =\n Dropdown.dropdown\n (Maybe.withDefault Dropdown.initialState (Dict.get folder.folder model.conflictDropdowns))\n { options = [ Dropdown.alignMenuRight ]\n , toggleMsg = ConflictDropdownMsg folder.folder\n , toggleButton =\n Dropdown.toggle\n [ Button.roleLink ]\n [ span [ class \"fas\", class <| conflictStrategyToIconName folder.conflictStrategy ] [] ]\n , items =\n [ Dropdown.buttonItem\n [ onClick (ConflictStrategyToggled folder.folder \"ignore\") ]\n [ span [ class \"fas fa-md fa-eject\" ] [], text \" Ignore\" ]\n , Dropdown.buttonItem\n [ onClick (ConflictStrategyToggled folder.folder \"marker\") ]\n [ span [ class \"fas fa-md fa-marker\" ] [], text \" Marker\" ]\n , Dropdown.buttonItem\n [ onClick (ConflictStrategyToggled folder.folder \"embrace\") ]\n [ span [ class \"fas fa-md fa-handshake\" ] [], text \" Embrace\" ]\n , Dropdown.buttonItem\n [ onClick (ConflictStrategyToggled folder.folder \"\") ]\n [ span [ class \"fas fa-md fa-eraser\" ] [], text \" Default\" ]\n ]\n }\n\n\nviewFolder : Model -> Commands.Folder -> Table.Row Msg\nviewFolder model folder =\n Table.tr []\n [ Table.td\n []\n [ span [ class \"fas fa-md fa-folder text-muted\" ] [] ]\n , Table.td\n []\n [ text folder.folder ]\n , Table.td\n []\n [ viewConflictDropdown model folder ]\n , Table.td\n []\n [ Util.viewToggleSwitch (ReadOnlyChanged folder.folder) \"\" folder.readOnly False ]\n , Table.td\n []\n [ Button.button\n [ Button.attrs [ class \"close\", onClick <| FolderRemove folder.folder ] ]\n [ span [ class \"fas fa-xs fa-times text-muted\" ] []\n ]\n ]\n ]\n\n\nviewFolders : Model -> Commands.Remote -> Html Msg\nviewFolders model remote =\n Table.table\n { options =\n [ Table.hover\n , Table.attr (class \"borderless-table\")\n ]\n , thead =\n Table.thead []\n [ Table.tr []\n [ Table.th\n [ Table.cellAttr (style \"width\" \"5%\") ]\n [ text \"\" ]\n , Table.th\n [ Table.cellAttr (style \"width\" \"55%\") ]\n [ span [ class \"text-muted small\" ] [ text \"Name\" ] ]\n , Table.th\n [ Table.cellAttr (style \"width\" \"20%\") ]\n [ span [ class \"text-muted small\" ] [ text \"Conflict Strategy\" ] ]\n , Table.th\n [ Table.cellAttr (style \"width\" \"15%\") ]\n [ span [ class \"text-muted small\" ] [ text \"Read Only?\" ] ]\n , Table.th\n [ Table.cellAttr (style \"width\" \"5%\") ]\n []\n ]\n ]\n , tbody =\n Table.tbody []\n (List.map\n (\\f -> viewFolder model f)\n remote.folders\n )\n }\n\n\nviewMaybeFolders : Model -> Commands.Remote -> Html Msg\nviewMaybeFolders model remote =\n let\n folders =\n LE.uniqueBy .folder remote.folders\n in\n if List.length folders <= 0 then\n span\n [ class \"text-muted text-center\" ]\n [ text \"No folders. This means this user can see everthing.\"\n , br [] []\n , text \"Add a new folder below to limit what this remote can see.\"\n , br [] []\n , br [] []\n ]\n\n else\n div []\n [ viewFolders model remote\n , br [] []\n , hr [] []\n ]\n\n\nviewRemoteFoldersContent : Model -> List (Grid.Column Msg)\nviewRemoteFoldersContent model =\n [ Grid.col [ Col.xs12 ]\n [ h4 [] [ span [ class \"text-muted text-center\" ] [ text \"Visible folders\" ] ]\n , viewMaybeFolders model model.remote\n , br [] []\n , br [] []\n , h4 [] [ span [ class \"text-muted text-center\" ] [ text \"All folders\" ] ]\n , MoveCopy.viewSearchBox SearchInput model.filter\n , MoveCopy.viewDirList DirChosen model.filter model.allDirs\n , case model.state of\n Ready ->\n text \"\"\n\n Fail message ->\n Util.buildAlert model.alert AlertMsg Alert.danger \"Oh no!\" (\"Could not add remote: \" ++ message)\n ]\n ]\n\n\nview : Model -> Html Msg\nview model =\n Modal.config ModalClose\n |> Modal.large\n |> Modal.withAnimation AnimateModal\n |> Modal.header [ class \"modal-title modal-header-primary\" ]\n [ h4 [] [ text \"Edit folders of »\", text model.remote.name, text \"«\" ] ]\n |> Modal.body []\n [ Grid.containerFluid []\n [ Grid.row\n [ Row.attrs [ style \"min-width\" \"60vh\", class \"scrollable-modal-row\" ]\n ]\n (viewRemoteFoldersContent model)\n ]\n ]\n |> Modal.footer []\n [ Button.button\n [ Button.outlinePrimary\n , Button.attrs [ onClick <| AnimateModal Modal.hiddenAnimated ]\n ]\n [ text \"Close\" ]\n ]\n |> Modal.view model.modal\n\n\nshow : Commands.Remote -> Msg\nshow remote =\n ModalShow remote\n\n\n\n-- SUBSCRIPTIONS\n\n\nsubscriptions : Model -> Sub Msg\nsubscriptions model =\n Sub.batch\n [ Modal.subscriptions model.modal AnimateModal\n , Alert.subscriptions model.alert AlertMsg\n , Sub.batch\n (List.map\n (\\( name, state ) -> Dropdown.subscriptions state (ConflictDropdownMsg name))\n (Dict.toList model.conflictDropdowns)\n )\n ]\n" }, { "path": "gateway/elm/src/Modals/RemoteRemove.elm", "content": "module Modals.RemoteRemove exposing (Model, Msg, newModel, show, subscriptions, update, view)\n\nimport Bootstrap.Alert as Alert\nimport Bootstrap.Button as Button\n\nimport Bootstrap.Grid as Grid\nimport Bootstrap.Grid.Col as Col\nimport Bootstrap.Modal as Modal\nimport Browser.Events as Events\nimport Commands\nimport Html exposing (..)\nimport Html.Attributes exposing (..)\nimport Html.Events exposing (..)\nimport Http\nimport Json.Decode as D\n\nimport Util\n\n\ntype State\n = Ready\n | Fail String\n\n\ntype alias Model =\n { state : State\n , name : String\n , modal : Modal.Visibility\n , alert : Alert.Visibility\n }\n\n\ntype Msg\n = DoRemove\n | ModalShow String\n | GotResponse (Result Http.Error String)\n | AnimateModal Modal.Visibility\n | AlertMsg Alert.Visibility\n | ModalClose\n | KeyPress String\n\n\n\n-- INIT\n\n\nnewModel : Model\nnewModel =\n newModelWithState \"\" Modal.hidden\n\n\nnewModelWithState : String -> Modal.Visibility -> Model\nnewModelWithState name state =\n { state = Ready\n , modal = state\n , name = name\n , alert = Alert.shown\n }\n\n\n\n-- UPDATE\n\n\nsubmit : Model -> Cmd Msg\nsubmit model =\n Commands.doRemoteRemove GotResponse model.name\n\n\nupdate : Msg -> Model -> ( Model, Cmd Msg )\nupdate msg model =\n case msg of\n DoRemove ->\n ( model, submit model )\n\n GotResponse result ->\n case result of\n Ok _ ->\n ( { model | modal = Modal.hidden }, Cmd.none )\n\n Err err ->\n ( { model | state = Fail <| Util.httpErrorToString err }, Cmd.none )\n\n AnimateModal visibility ->\n ( { model | modal = visibility }, Cmd.none )\n\n ModalShow path ->\n ( newModelWithState path Modal.shown, Cmd.none )\n\n ModalClose ->\n ( { model | modal = Modal.hidden }, Cmd.none )\n\n AlertMsg vis ->\n ( { model | alert = vis }, Cmd.none )\n\n KeyPress key ->\n if model.modal == Modal.hidden then\n ( model, Cmd.none )\n\n else\n case key of\n \"Enter\" ->\n ( model, submit model )\n\n _ ->\n ( model, Cmd.none )\n\n\n\n-- VIEW\n\n\nviewRemoteAddContent : Model -> List (Grid.Column Msg)\nviewRemoteAddContent model =\n [ Grid.col [ Col.xs12 ]\n [ text\n (\"Removing »\"\n ++ model.name\n ++ \"« cannot be reverted. If you are the last one caching the data of this remote,\"\n ++ \" the data might vanish forever and cannot be restored.\"\n )\n ]\n ]\n\n\nview : Model -> Html Msg\nview model =\n Modal.config ModalClose\n |> Modal.large\n |> Modal.withAnimation AnimateModal\n |> Modal.header [ class \"modal-title modal-header-danger\" ]\n [ h4 [] [ text \"Really remove?\" ] ]\n |> Modal.body []\n [ Grid.containerFluid []\n [ Grid.row [] (viewRemoteAddContent model) ]\n ]\n |> Modal.footer []\n [ Button.button\n [ Button.danger\n , Button.attrs\n [ onClick DoRemove\n , type_ \"submit\"\n ]\n ]\n [ text \"Remove\" ]\n , Button.button\n [ Button.outlinePrimary\n , Button.attrs [ onClick <| AnimateModal Modal.hiddenAnimated ]\n ]\n [ text \"Cancel\" ]\n ]\n |> Modal.view model.modal\n\n\nshow : String -> Msg\nshow name =\n ModalShow name\n\n\n\n-- SUBSCRIPTIONS\n\n\nsubscriptions : Model -> Sub Msg\nsubscriptions model =\n Sub.batch\n [ Modal.subscriptions model.modal AnimateModal\n , Alert.subscriptions model.alert AlertMsg\n , Events.onKeyPress (D.map KeyPress <| D.field \"key\" D.string)\n ]\n" }, { "path": "gateway/elm/src/Modals/Remove.elm", "content": "module Modals.Remove exposing (Model, Msg, newModel, show, subscriptions, update, view)\n\nimport Bootstrap.Alert as Alert\nimport Bootstrap.Button as Button\nimport Bootstrap.Grid as Grid\nimport Bootstrap.Grid.Col as Col\nimport Bootstrap.Modal as Modal\nimport Browser.Events as Events\nimport Commands\nimport Html exposing (..)\nimport Html.Attributes exposing (..)\nimport Html.Events exposing (..)\nimport Http\nimport Json.Decode as D\nimport Util\n\n\ntype State\n = Ready\n | Fail String\n\n\ntype alias Model =\n { state : State\n , modal : Modal.Visibility\n , alert : Alert.Visibility\n , selected : List String\n }\n\n\ntype Msg\n = RemoveAll (List String)\n | ModalShow (List String)\n | GotResponse (Result Http.Error String)\n | AnimateModal Modal.Visibility\n | AlertMsg Alert.Visibility\n | ModalClose\n | KeyPress String\n\n\n\n-- INIT\n\n\nnewModel : Model\nnewModel =\n { state = Ready\n , modal = Modal.hidden\n , alert = Alert.shown\n , selected = []\n }\n\n\n\n-- UPDATE\n\n\nupdate : Msg -> Model -> ( Model, Cmd Msg )\nupdate msg model =\n case msg of\n RemoveAll paths ->\n ( model, Commands.doRemove GotResponse paths )\n\n GotResponse result ->\n case result of\n Ok _ ->\n -- New list model means also new checked entries.\n ( { model | state = Ready, modal = Modal.hidden }, Cmd.none )\n\n Err err ->\n ( { model | state = Fail <| Util.httpErrorToString err }, Cmd.none )\n\n AnimateModal visibility ->\n ( { model | modal = visibility }, Cmd.none )\n\n ModalShow paths ->\n ( { model | modal = Modal.shown, selected = paths }, Cmd.none )\n\n ModalClose ->\n ( { model | modal = Modal.hidden, state = Ready }, Cmd.none )\n\n AlertMsg vis ->\n ( { model | alert = vis }, Cmd.none )\n\n KeyPress key ->\n ( model\n , if model.modal == Modal.hidden then\n Cmd.none\n\n else\n case key of\n \"Enter\" ->\n Commands.doRemove GotResponse model.selected\n\n _ ->\n Cmd.none\n )\n\n\n\n-- VIEW\n\n\npluralizeItems : Int -> String\npluralizeItems count =\n if count == 1 then\n \"item\"\n\n else\n \"items\"\n\n\nviewRemoveContent : Model -> Int -> List (Grid.Column Msg)\nviewRemoveContent model nSelected =\n [ Grid.col [ Col.xs12 ]\n [ case model.state of\n Ready ->\n text (\"This would remove the \" ++ String.fromInt nSelected ++ \" selected \" ++ pluralizeItems nSelected ++ \".\")\n\n Fail message ->\n Util.buildAlert model.alert AlertMsg Alert.danger \"Oh no!\" (\"Could not remove directory: \" ++ message)\n ]\n ]\n\n\nview : Model -> List String -> Html Msg\nview model selectedPaths =\n Modal.config ModalClose\n |> Modal.large\n |> Modal.withAnimation AnimateModal\n |> Modal.header [ class \"modal-title modal-header-warning\" ]\n [ h4 [] [ text \"Really remove?\" ] ]\n |> Modal.body []\n [ Grid.containerFluid []\n [ Grid.row [] (viewRemoveContent model (List.length selectedPaths)) ]\n ]\n |> Modal.footer []\n [ Button.button\n [ Button.warning\n , Button.attrs\n [ onClick <| RemoveAll selectedPaths\n , disabled\n (case model.state of\n Fail _ ->\n True\n\n _ ->\n False\n )\n ]\n ]\n [ text \"Remove\" ]\n , Button.button\n [ Button.outlinePrimary\n , Button.attrs [ onClick <| AnimateModal Modal.hiddenAnimated ]\n ]\n [ text \"Cancel\" ]\n ]\n |> Modal.view model.modal\n\n\nshow : List String -> Msg\nshow paths =\n ModalShow paths\n\n\n\n-- SUBSCRIPTIONS\n\n\nsubscriptions : Model -> Sub Msg\nsubscriptions model =\n Sub.batch\n [ Modal.subscriptions model.modal AnimateModal\n , Alert.subscriptions model.alert AlertMsg\n , Events.onKeyPress (D.map KeyPress <| D.field \"key\" D.string)\n ]\n" }, { "path": "gateway/elm/src/Modals/Rename.elm", "content": "module Modals.Rename exposing (Model, Msg, newModel, show, subscriptions, update, view)\n\nimport Bootstrap.Alert as Alert\nimport Bootstrap.Button as Button\nimport Bootstrap.Form.Input as Input\nimport Bootstrap.Grid as Grid\nimport Bootstrap.Grid.Col as Col\nimport Bootstrap.Modal as Modal\nimport Browser.Events as Events\nimport Commands\nimport Html exposing (..)\nimport Html.Attributes exposing (..)\nimport Html.Events exposing (..)\nimport Http\nimport Json.Decode as D\nimport Util\n\n\ntype State\n = Ready\n | Fail String\n\n\ntype alias Model =\n { state : State\n , currPath : String\n , inputName : String\n , modal : Modal.Visibility\n , alert : Alert.Visibility\n }\n\n\ntype Msg\n = DoRename\n | InputChanged String\n | ModalShow String\n | GotResponse (Result Http.Error String)\n | AnimateModal Modal.Visibility\n | AlertMsg Alert.Visibility\n | ModalClose\n | KeyPress String\n\n\n\n-- INIT\n\n\nnewModel : Model\nnewModel =\n { state = Ready\n , modal = Modal.hidden\n , inputName = \"\"\n , currPath = \"\"\n , alert = Alert.shown\n }\n\n\n\n-- UPDATE\n\n\ntriggerRename : String -> String -> Cmd Msg\ntriggerRename sourcePath newName =\n Commands.doMove\n GotResponse\n sourcePath\n (Util.joinPath [ Util.dirname sourcePath, Util.basename newName ])\n\n\nupdate : Msg -> Model -> ( Model, Cmd Msg )\nupdate msg model =\n case msg of\n DoRename ->\n ( model, triggerRename model.currPath model.inputName )\n\n InputChanged inputName ->\n ( { model | inputName = inputName }, Cmd.none )\n\n GotResponse result ->\n case result of\n Ok _ ->\n -- New list model means also new checked entries.\n ( { model | state = Ready, modal = Modal.hidden }, Cmd.none )\n\n Err err ->\n ( { model | state = Fail <| Util.httpErrorToString err }, Cmd.none )\n\n AnimateModal visibility ->\n ( { model | modal = visibility }, Cmd.none )\n\n ModalShow currPath ->\n ( { model | modal = Modal.shown, inputName = \"\", currPath = currPath }, Cmd.none )\n\n ModalClose ->\n ( { model | modal = Modal.hidden, state = Ready }, Cmd.none )\n\n AlertMsg vis ->\n ( { model | alert = vis }, Cmd.none )\n\n KeyPress key ->\n if model.modal == Modal.hidden then\n ( model, Cmd.none )\n\n else\n case key of\n \"Enter\" ->\n ( model, triggerRename model.currPath model.inputName )\n\n _ ->\n ( model, Cmd.none )\n\n\n\n-- VIEW\n\n\nviewRenameContent : Model -> List (Grid.Column Msg)\nviewRenameContent model =\n [ Grid.col [ Col.xs12 ]\n [ Input.text\n [ Input.id \"rename-input\"\n , Input.large\n , Input.placeholder \"New name\"\n , Input.onInput InputChanged\n , Input.attrs [ autofocus True ]\n ]\n , br [] []\n , case model.state of\n Ready ->\n text \"\"\n\n Fail message ->\n Util.buildAlert\n model.alert\n AlertMsg\n Alert.danger\n \"Oh no!\"\n (\"Could not rename path: \" ++ message)\n ]\n ]\n\n\nview : Model -> Html Msg\nview model =\n Modal.config ModalClose\n |> Modal.large\n |> Modal.withAnimation AnimateModal\n |> Modal.header [ class \"modal-title modal-header-primary\" ]\n [ h4 []\n [ text \"Rename \"\n , span []\n [ text \"»\"\n , text (Util.basename model.currPath)\n , text \"«\"\n ]\n , if String.length model.inputName > 0 then\n span []\n [ text \" to \"\n , span []\n [ text \"»\"\n , text model.inputName\n , text \"«\"\n ]\n ]\n\n else\n text \"\"\n ]\n ]\n |> Modal.body []\n [ Grid.containerFluid []\n [ Grid.row [] (viewRenameContent model) ]\n ]\n |> Modal.footer []\n [ Button.button\n [ Button.primary\n , Button.attrs\n [ onClick DoRename\n , type_ \"submit\"\n , disabled\n (String.length model.inputName\n == 0\n || (case model.state of\n Fail _ ->\n True\n\n _ ->\n False\n )\n )\n ]\n ]\n [ text \"Rename\" ]\n , Button.button\n [ Button.outlinePrimary\n , Button.attrs [ onClick <| AnimateModal Modal.hiddenAnimated ]\n ]\n [ text \"Cancel\" ]\n ]\n |> Modal.view model.modal\n\n\nshow : String -> Msg\nshow currPath =\n ModalShow currPath\n\n\n\n-- SUBSCRIPTIONS\n\n\nsubscriptions : Model -> Sub Msg\nsubscriptions model =\n Sub.batch\n [ Modal.subscriptions model.modal AnimateModal\n , Alert.subscriptions model.alert AlertMsg\n , Events.onKeyPress (D.map KeyPress <| D.field \"key\" D.string)\n ]\n" }, { "path": "gateway/elm/src/Modals/Share.elm", "content": "module Modals.Share exposing (Model, Msg, newModel, show, subscriptions, update, view)\n\nimport Bootstrap.Button as Button\nimport Bootstrap.Grid as Grid\nimport Bootstrap.Grid.Col as Col\nimport Bootstrap.Grid.Row as Row\nimport Bootstrap.Modal as Modal\nimport Html exposing (..)\nimport Html.Attributes exposing (..)\nimport Html.Events exposing (..)\nimport Url\nimport Util\n\n\ntype alias Model =\n { paths : List String\n , modal : Modal.Visibility\n }\n\n\ntype Msg\n = ModalShow (List String)\n | AnimateModal Modal.Visibility\n | ModalClose\n\n\n\n-- INIT\n\n\nnewModel : Model\nnewModel =\n { paths = []\n , modal = Modal.hidden\n }\n\n\n\n-- UPDATE\n\n\nupdate : Msg -> Model -> ( Model, Cmd Msg )\nupdate msg model =\n case msg of\n AnimateModal visibility ->\n ( { model | modal = visibility }, Cmd.none )\n\n ModalShow paths ->\n ( { model | modal = Modal.shown, paths = paths }, Cmd.none )\n\n ModalClose ->\n ( { model | modal = Modal.hidden, paths = [] }, Cmd.none )\n\n\n\n-- VIEW\n\n\nformatEntry : Url.Url -> String -> Html msg\nformatEntry url path =\n let\n link =\n Util.urlPrefixToString url ++ \"get\" ++ Util.urlEncodePath path\n in\n li [] [ a [ href link ] [ text link ] ]\n\n\nviewShare : Model -> Url.Url -> List (Grid.Column Msg)\nviewShare model url =\n [ Grid.col [ Col.xs12 ]\n [ p [] [ text \"Use those links to share the selected files with people that do not use brig.\" ]\n , p [] [ b [] [ text \"Note:\" ], text \" Remember, they still need to authenticate themselves.\" ]\n , ul [ id \"share-list\" ] (List.map (formatEntry url) model.paths)\n ]\n ]\n\n\nview : Model -> Url.Url -> Html Msg\nview model url =\n Modal.config ModalClose\n |> Modal.large\n |> Modal.withAnimation AnimateModal\n |> Modal.header [ class \"modal-title modal-header-primary\" ]\n [ h4 [] [ text \"Share hyperlinks\" ] ]\n |> Modal.body []\n [ Grid.containerFluid []\n [ Grid.row [ Row.attrs [ class \"scrollable-modal-row\" ] ] (viewShare model url) ]\n ]\n |> Modal.footer []\n [ Button.button\n [ Button.outlinePrimary\n , Button.attrs [ onClick <| AnimateModal Modal.hiddenAnimated ]\n ]\n [ text \"Close\" ]\n ]\n |> Modal.view model.modal\n\n\nshow : List String -> Msg\nshow paths =\n ModalShow paths\n\n\n\n-- SUBSCRIPTIONS\n\n\nsubscriptions : Model -> Sub Msg\nsubscriptions model =\n Sub.batch\n [ Modal.subscriptions model.modal AnimateModal\n ]\n" }, { "path": "gateway/elm/src/Modals/Upload.elm", "content": "module Modals.Upload exposing\n ( Model\n , Msg\n , buildButton\n , newModel\n , subscriptions\n , update\n , viewUploadState\n )\n\nimport Bootstrap.Alert as Alert\nimport Bootstrap.Button as Button\nimport Bootstrap.Grid as Grid\nimport Bootstrap.Grid.Col as Col\nimport Bootstrap.Progress as Progress\nimport Bootstrap.Text as Text\nimport Commands\nimport Delay\nimport Dict\nimport File\nimport Html exposing (..)\nimport Html.Attributes exposing (..)\nimport Html.Events exposing (..)\nimport Http\nimport Json.Decode as D\nimport List\nimport Tuple\n\n\ntype alias Alertable =\n { alert : Alert.Visibility\n , path : String\n }\n\n\ntype alias Model =\n { uploads : Dict.Dict String Float\n , failed : List Alertable\n , success : List Alertable\n }\n\n\ntype Msg\n = UploadSelectedFiles String (List File.File)\n | UploadProgress String Http.Progress\n | Uploaded String (Result Http.Error ())\n | UploadCancel String\n | AlertMsg String Alert.Visibility\n\n\n\n-- INIT\n\n\nnewModel : Model\nnewModel =\n { uploads = Dict.empty\n , failed = []\n , success = []\n }\n\n\nalertMapper : String -> Alert.Visibility -> Alertable -> Alertable\nalertMapper path vis a =\n case a.path == path of\n True ->\n { a | alert = vis }\n\n False ->\n a\n\n\n\n-- UPDATE\n\n\nupdate : Msg -> Model -> ( Model, Cmd Msg )\nupdate msg model =\n case msg of\n UploadSelectedFiles root files ->\n let\n newUploads =\n Dict.union model.uploads <| Dict.fromList (List.map (\\f -> ( File.name f, 0 )) files)\n in\n ( { model | uploads = newUploads }\n , Cmd.batch (List.map (Commands.doUpload Uploaded root) files)\n )\n\n UploadProgress path progress ->\n case progress of\n Http.Sending p ->\n ( { model | uploads = Dict.insert path (Http.fractionSent p) model.uploads }, Cmd.none )\n\n Http.Receiving _ ->\n ( model, Cmd.none )\n\n Uploaded path result ->\n let\n newUploads =\n Dict.remove path model.uploads\n in\n case result of\n Ok _ ->\n ( { model\n | uploads = newUploads\n , success = Alertable Alert.shown path :: model.success\n }\n , Delay.after 5 Delay.Second (AlertMsg path Alert.closed)\n )\n\n Err _ ->\n ( { model\n | uploads = newUploads\n , failed = Alertable Alert.shown path :: model.failed\n }\n , Delay.after 30 Delay.Second (AlertMsg path Alert.closed)\n )\n\n UploadCancel path ->\n ( { model | uploads = Dict.remove path model.uploads }\n , Http.cancel (\"upload-\" ++ path)\n )\n\n AlertMsg path vis ->\n ( { model\n | success = List.map (alertMapper path vis) model.success\n , failed = List.map (alertMapper path vis) model.failed\n }\n , Cmd.none\n )\n\n\n\n-- VIEW\n\n\nfilesDecoder : D.Decoder (List File.File)\nfilesDecoder =\n D.at [ \"target\", \"files\" ] (D.list File.decoder)\n\n\nbuildButton : Model -> Bool -> String -> (Msg -> msg) -> Html msg\nbuildButton model currIsFile currRoot toMsg =\n label\n [ class \"btn btn-file btn-link btn-default text-left\"\n , id \"action-btn\"\n , if currIsFile then\n class \"disabled\"\n\n else\n class \"btn-default\"\n ]\n [ span [ class \"fas fa-plus\" ] []\n , span [ class \"d-lg-inline d-none\" ] [ text \"\\u{00A0}\\u{00A0}Upload\" ]\n , input\n [ type_ \"file\"\n , multiple True\n , on \"change\"\n (D.map toMsg\n (D.map\n (UploadSelectedFiles currRoot)\n filesDecoder\n )\n )\n , style \"display\" \"none\"\n , disabled currIsFile\n ]\n []\n ]\n\n\nclampText : String -> Int -> String\nclampText text length =\n if String.length text <= length then\n text\n\n else\n String.slice 0 length text ++ \"…\"\n\n\nviewAlert : Alert.Visibility -> String -> Bool -> Html Msg\nviewAlert alert path isSuccess =\n Alert.config\n |> Alert.dismissableWithAnimation (AlertMsg path)\n |> (if isSuccess then\n Alert.success\n\n else\n Alert.danger\n )\n |> Alert.children\n [ Grid.row []\n [ Grid.col [ Col.xs10 ]\n [ span\n [ if isSuccess then\n class \"fas fa-xs fa-check\"\n\n else\n class \"fas fa-xs fa-exclamation-circle\"\n ]\n []\n , text (\" \" ++ clampText path 15)\n ]\n , Grid.col [ Col.xs2, Col.textAlign Text.alignXsRight ]\n [ Button.button\n [ Button.roleLink\n , Button.attrs\n [ class \"notification-close-btn\"\n , onClick (AlertMsg path Alert.closed)\n ]\n ]\n [ span [ class \"fas fa-xs fa-times\" ] [] ]\n ]\n ]\n ]\n |> Alert.view alert\n\n\nviewProgressIndicator : String -> Float -> Html Msg\nviewProgressIndicator path fraction =\n Grid.row []\n [ Grid.col [ Col.md10 ]\n [ Progress.progress\n [ Progress.value (100 * fraction)\n , Progress.customLabel [ text (clampText path 25) ]\n , Progress.attrs [ style \"height\" \"25px\" ]\n , Progress.wrapperAttrs [ style \"height\" \"25px\" ]\n ]\n ]\n , Grid.col [ Col.md2 ]\n [ Button.button\n [ Button.roleLink\n , Button.attrs [ class \"progress-cancel\", onClick (UploadCancel path) ]\n ]\n [ span [ class \"fas fa-xs fa-times\" ] [] ]\n ]\n ]\n\n\nviewUploadState : Model -> Html Msg\nviewUploadState model =\n div []\n [ br [] []\n , br [] []\n , ul [ class \"notification-list list-group\" ]\n (List.map (\\a -> viewAlert a.alert a.path True) model.success\n ++ List.map (\\a -> viewAlert a.alert a.path False) model.failed\n ++ List.map (\\p -> viewProgressIndicator (Tuple.first p) (Tuple.second p)) (Dict.toList model.uploads)\n )\n ]\n\n\n\n-- SUBSCRIPTIONS\n\n\nsubscriptions : Model -> Sub Msg\nsubscriptions model =\n Sub.batch\n [ Sub.batch\n (List.map\n (\\p -> Http.track (\"upload-\" ++ p) (UploadProgress p))\n (Dict.keys model.uploads)\n )\n , Sub.batch (List.map (\\a -> Alert.subscriptions a.alert (AlertMsg a.path)) model.success)\n , Sub.batch (List.map (\\a -> Alert.subscriptions a.alert (AlertMsg a.path)) model.failed)\n ]\n" }, { "path": "gateway/elm/src/Pinger.elm", "content": "port module Pinger exposing (pinger)\n\n\nport pinger : (String -> msg) -> Sub msg\n" }, { "path": "gateway/elm/src/Routes/Commits.elm", "content": "module Routes.Commits exposing\n ( Model\n , Msg\n , newModel\n , reload\n , reloadIfNeeded\n , subscriptions\n , update\n , updateUrl\n , view\n )\n\nimport Bootstrap.Alert as Alert\nimport Bootstrap.Button as Button\nimport Bootstrap.Form.Input as Input\nimport Bootstrap.Form.InputGroup as InputGroup\nimport Bootstrap.Grid as Grid\nimport Bootstrap.Grid.Col as Col\nimport Bootstrap.Grid.Row as Row\nimport Bootstrap.ListGroup as ListGroup\nimport Bootstrap.Table as Table\nimport Bootstrap.Text as Text\nimport Browser.Navigation as Nav\nimport Commands\nimport Delay\nimport Dict\nimport Html exposing (..)\nimport Html.Attributes exposing (..)\nimport Html.Events exposing (..)\nimport Html.Lazy as Lazy\nimport Http\nimport Scroll\nimport Time\nimport Url\nimport Util\n\n\n\n-- MODEL:\n\n\nloadLimit : Int\nloadLimit =\n 20\n\n\ntype State\n = Loading\n | Failure String\n | Success (List Commands.Commit)\n\n\ntype alias Model =\n { key : Nav.Key\n , state : State\n , zone : Time.Zone\n , filter : String\n , offset : Int\n , alert : Util.AlertState\n , url : Url.Url\n , haveStagedChanges : Bool\n , rights : List String\n }\n\n\nnewModel : Url.Url -> Nav.Key -> Time.Zone -> List String -> Model\nnewModel url key zone rights =\n { key = key\n , state = Loading\n , zone = zone\n , filter = \"\"\n , offset = 0\n , alert = Util.defaultAlertState\n , url = url\n , haveStagedChanges = False\n , rights = rights\n }\n\n\nupdateUrl : Model -> Url.Url -> Model\nupdateUrl model url =\n { model | url = url }\n\n\n\n-- MESSAGES:\n\n\ntype Msg\n = GotLogResponse Bool (Result Http.Error Commands.Log)\n | GotResetResponse (Result Http.Error String)\n | CheckoutClicked String\n | SearchInput String\n | OnScroll Scroll.ScreenData\n | AlertMsg Alert.Visibility\n\n\n\n-- UPDATE:\n\n\nreload : Model -> Cmd Msg\nreload model =\n Commands.doLog (GotLogResponse True) 0 (model.offset + loadLimit) model.filter\n\n\nreloadIfNeeded : Model -> Cmd Msg\nreloadIfNeeded model =\n case model.state of\n Success commits ->\n if List.length commits == 0 then\n reload model\n\n else\n Cmd.none\n\n _ ->\n Cmd.none\n\n\nreloadWithoutFlush : Model -> Int -> Cmd Msg\nreloadWithoutFlush model newOffset =\n Commands.doLog (GotLogResponse False) newOffset loadLimit model.filter\n\n\ntoMap : List Commands.Commit -> Dict.Dict Int Commands.Commit\ntoMap commits =\n Dict.fromList (List.map (\\c -> ( c.index, c )) commits)\n\n\nmergeCommits : List Commands.Commit -> List Commands.Commit -> List Commands.Commit\nmergeCommits old new =\n Dict.union (toMap new) (toMap old)\n |> Dict.toList\n |> List.map (\\( _, v ) -> v)\n |> List.reverse\n\n\nshowAlert : Model -> Float -> Util.AlertType -> String -> ( Model, Cmd Msg )\nshowAlert model duration modalTyp message =\n let\n newAlert =\n Util.AlertState message modalTyp Alert.shown\n in\n ( { model | alert = newAlert }\n , Cmd.batch\n [ Delay.after duration Delay.Second (AlertMsg Alert.closed) ]\n )\n\n\nupdate : Msg -> Model -> ( Model, Cmd Msg )\nupdate msg model =\n case msg of\n GotLogResponse doFlush result ->\n case result of\n Ok log ->\n -- Got a new load of data. Merge it with the previous dataset,\n -- unless we want to flush the current view.\n let\n ( prevCommits, newOffset ) =\n if doFlush then\n ( [], 0 )\n\n else\n case model.state of\n Success oldCommits ->\n ( oldCommits, model.offset + loadLimit )\n\n _ ->\n ( [], model.offset )\n in\n ( { model\n | state = Success (mergeCommits prevCommits log.commits)\n , offset = newOffset\n , haveStagedChanges = log.haveStagedChanges\n }\n , Cmd.none\n )\n\n Err err ->\n ( { model | state = Failure (Util.httpErrorToString err) }, Cmd.none )\n\n GotResetResponse result ->\n case result of\n Ok _ ->\n showAlert model 5 Util.Success \"Succesfully reset state.\"\n\n Err err ->\n showAlert model 15 Util.Danger (\"Failed to reset: \" ++ Util.httpErrorToString err)\n\n CheckoutClicked hash ->\n ( model, Commands.doReset GotResetResponse \"/\" hash )\n\n SearchInput filter ->\n let\n upModel =\n { model | filter = filter }\n in\n ( upModel, reload upModel )\n\n OnScroll data ->\n if String.startsWith \"/log\" model.url.path then\n if Scroll.hasHitBottom data then\n ( model, reloadWithoutFlush model (model.offset + loadLimit) )\n\n else\n -- We don't need to reload yet.\n ( model, Cmd.none )\n\n else\n -- We're currently not visible. Forget updating.\n ( model, Cmd.none )\n\n AlertMsg vis ->\n let\n newAlert =\n Util.AlertState model.alert.message model.alert.typ vis\n in\n ( { model | alert = newAlert }, Cmd.none )\n\n\n\n-- VIEW:\n\n\nviewSearchBox : Model -> Html Msg\nviewSearchBox model =\n InputGroup.config\n (InputGroup.text\n [ Input.placeholder \"Search\"\n , Input.attrs\n [ onInput SearchInput\n , value model.filter\n ]\n ]\n )\n |> InputGroup.successors\n [ InputGroup.span [ class \"input-group-addon\" ]\n [ button [] [ span [ class \"fas fa-search fa-xs input-group-addon\" ] [] ]\n ]\n ]\n |> InputGroup.attrs [ class \"stylish-input-group input-group\" ]\n |> InputGroup.view\n\n\nviewCommit : Model -> Commands.Commit -> ListGroup.Item Msg\nviewCommit model commit =\n ListGroup.li []\n [ Grid.row\n []\n [ Grid.col\n [ Col.xs1\n , Col.textAlign Text.alignXsLeft\n ]\n [ span [ class \"fas fa-lg fa-save text-xs-right\" ] []\n ]\n , Grid.col [ Col.xs8, Col.textAlign Text.alignXsLeft ]\n [ text commit.msg\n ]\n , Grid.col\n [ Col.xs3\n , Col.textAlign Text.alignXsRight\n ]\n [ Button.button\n [ Button.outlineDanger\n , Button.attrs\n [ onClick <| CheckoutClicked commit.hash\n , disabled\n ((not model.haveStagedChanges && List.member \"head\" commit.tags)\n || not (List.member \"fs.edit\" model.rights)\n )\n ]\n ]\n [ text \"Checkout\" ]\n ]\n ]\n ]\n\n\nviewCommitList : Model -> List Commands.Commit -> Html Msg\nviewCommitList model commits =\n ListGroup.ul (List.map (viewCommit model) (List.filter (\\c -> String.length c.msg > 0) commits))\n\n\nviewCommitListContainer : Model -> List Commands.Commit -> Html Msg\nviewCommitListContainer model commits =\n Grid.row []\n [ Grid.col [ Col.lg2, Col.attrs [ class \"d-none d-lg-block\" ] ] []\n , Grid.col [ Col.lg8, Col.md12 ]\n [ h4 [ class \"text-muted text-center\" ] [ text \"Commits\" ]\n , Util.viewAlert AlertMsg model.alert\n , br [] []\n , viewCommitList model commits\n , br [] []\n ]\n , Grid.col [ Col.lg2, Col.attrs [ class \"d-none d-lg-block\" ] ] []\n ]\n\n\nview : Model -> Html Msg\nview model =\n case model.state of\n Loading ->\n text \"Still loading\"\n\n Failure err ->\n text (\"Failed to load log: \" ++ err)\n\n Success commits ->\n Grid.row []\n [ Grid.col\n [ Col.lg12 ]\n [ Grid.row [ Row.attrs [ id \"main-header-row\" ] ]\n [ Grid.col [ Col.xl9 ] [ text \"\" ]\n , Grid.col [ Col.xl3 ] [ Lazy.lazy viewSearchBox model ]\n ]\n , Grid.row [ Row.attrs [ id \"main-content-row\" ] ]\n [ Grid.col\n [ Col.xl10 ]\n [ viewCommitListContainer model commits\n ]\n ]\n ]\n ]\n\n\n\n-- SUBSCRIPTIONS:\n\n\nsubscriptions : Model -> Sub Msg\nsubscriptions model =\n Scroll.scrollOrResize OnScroll\n" }, { "path": "gateway/elm/src/Routes/DeletedFiles.elm", "content": "module Routes.DeletedFiles exposing\n ( Model\n , Msg\n , newModel\n , reload\n , reloadIfNeeded\n , subscriptions\n , update\n , updateUrl\n , view\n )\n\nimport Bootstrap.Alert as Alert\nimport Bootstrap.Button as Button\nimport Bootstrap.Form.Input as Input\nimport Bootstrap.Form.InputGroup as InputGroup\nimport Bootstrap.Grid as Grid\nimport Bootstrap.Grid.Col as Col\nimport Bootstrap.Grid.Row as Row\nimport Bootstrap.Table as Table\nimport Bootstrap.Text as Text\nimport Browser.Navigation as Nav\nimport Commands\nimport Delay\nimport Dict\nimport Filesize\nimport Html exposing (..)\nimport Html.Attributes exposing (..)\nimport Html.Events exposing (..)\nimport Html.Lazy as Lazy\nimport Http\nimport Scroll\nimport Time\nimport Url\nimport Util\n\n\n\n-- MODEL:\n\n\nloadLimit : Int\nloadLimit =\n 25\n\n\ntype State\n = Loading\n | Failure String\n | Success (List Commands.Entry)\n\n\ntype alias Model =\n { key : Nav.Key\n , state : State\n , zone : Time.Zone\n , filter : String\n , offset : Int\n , alert : Util.AlertState\n , url : Url.Url\n , rights : List String\n }\n\n\nnewModel : Url.Url -> Nav.Key -> Time.Zone -> List String -> Model\nnewModel url key zone rights =\n { key = key\n , state = Loading\n , zone = zone\n , filter = \"\"\n , offset = 0\n , alert = Util.defaultAlertState\n , url = url\n , rights = rights\n }\n\n\nupdateUrl : Model -> Url.Url -> Model\nupdateUrl model url =\n { model | url = url }\n\n\n\n-- MESSAGES:\n\n\ntype Msg\n = GotDeletedPathsResponse Bool (Result Http.Error (List Commands.Entry))\n | GotUndeleteResponse (Result Http.Error String)\n | UndeleteClicked String\n | SearchInput String\n | AlertMsg Alert.Visibility\n | OnScroll Scroll.ScreenData\n\n\n\n-- UPDATE:\n\n\nreload : Model -> Cmd Msg\nreload model =\n Commands.doDeletedFiles (GotDeletedPathsResponse True) 0 (model.offset + loadLimit) model.filter\n\n\nreloadIfNeeded : Model -> Cmd Msg\nreloadIfNeeded model =\n case model.state of\n Success commits ->\n if List.length commits == 0 then\n reload model\n\n else\n Cmd.none\n\n _ ->\n Cmd.none\n\n\nreloadWithoutFlush : Model -> Int -> Cmd Msg\nreloadWithoutFlush model newOffset =\n Commands.doDeletedFiles (GotDeletedPathsResponse False) newOffset loadLimit model.filter\n\n\ntoMap : List Commands.Entry -> Dict.Dict String Commands.Entry\ntoMap entries =\n Dict.fromList (List.map (\\e -> ( e.path, e )) entries)\n\n\nsortEntries : Commands.Entry -> Commands.Entry -> Order\nsortEntries a b =\n let\n inv =\n \\v ->\n if v then\n 0\n\n else\n 1\n in\n case compare (inv a.isDir) (inv b.isDir) of\n EQ ->\n compare a.path b.path\n\n other ->\n other\n\n\nmergeEntries : List Commands.Entry -> List Commands.Entry -> List Commands.Entry\nmergeEntries old new =\n Dict.union (toMap new) (toMap old)\n |> Dict.toList\n |> List.map (\\( _, v ) -> v)\n |> List.sortWith sortEntries\n\n\nupdate : Msg -> Model -> ( Model, Cmd Msg )\nupdate msg model =\n case msg of\n GotDeletedPathsResponse doFlush result ->\n case result of\n Ok entries ->\n let\n ( prevEntries, newOffset ) =\n if doFlush then\n ( [], 0 )\n\n else\n case model.state of\n Success oldEntries ->\n ( oldEntries, model.offset + loadLimit )\n\n _ ->\n ( [], model.offset )\n in\n ( { model\n | state = Success (mergeEntries prevEntries entries)\n , offset = newOffset\n }\n , Cmd.none\n )\n\n Err err ->\n ( { model | state = Failure (Util.httpErrorToString err) }, Cmd.none )\n\n UndeleteClicked path ->\n ( model, Commands.doUndelete GotUndeleteResponse path )\n\n SearchInput filter ->\n let\n upModel =\n { model | filter = filter }\n in\n ( upModel, reload upModel )\n\n GotUndeleteResponse result ->\n case result of\n Ok _ ->\n let\n newAlert =\n Util.AlertState\n \"Succcesfully undeleted one item.\"\n Util.Success\n Alert.shown\n in\n ( { model | alert = newAlert }\n , Cmd.batch\n [ reload model\n , Delay.after 5 Delay.Second (AlertMsg Alert.closed)\n ]\n )\n\n Err err ->\n let\n newAlert =\n Util.AlertState\n (\"Failed to undelete: \" ++ Util.httpErrorToString err)\n Util.Danger\n Alert.shown\n in\n ( model\n , Cmd.batch\n [ reload model\n , Delay.after 15 Delay.Second (AlertMsg Alert.closed)\n ]\n )\n\n AlertMsg vis ->\n let\n newAlert =\n Util.AlertState model.alert.message model.alert.typ vis\n in\n ( { model | alert = newAlert }, Cmd.none )\n\n OnScroll data ->\n if String.startsWith \"/deleted\" model.url.path then\n if Scroll.hasHitBottom data then\n ( model, reloadWithoutFlush model (model.offset + loadLimit) )\n\n else\n ( model, Cmd.none )\n\n else\n -- We're currently not visible. Forget updating.\n ( model, Cmd.none )\n\n\n\n-- VIEW:\n\n\nviewSearchBox : Model -> Html Msg\nviewSearchBox model =\n InputGroup.config\n (InputGroup.text\n [ Input.placeholder \"Search\"\n , Input.attrs\n [ onInput SearchInput\n , value model.filter\n ]\n ]\n )\n |> InputGroup.successors\n [ InputGroup.span [ class \"input-group-addon\" ]\n [ button [] [ span [ class \"fas fa-search fa-xs input-group-addon\" ] [] ]\n ]\n ]\n |> InputGroup.attrs [ class \"stylish-input-group input-group\" ]\n |> InputGroup.view\n\n\nfilterEntries : String -> List Commands.Entry -> List Commands.Entry\nfilterEntries filter entries =\n case filter of\n \"\" ->\n entries\n\n _ ->\n List.filter (\\e -> String.contains filter e.path) entries\n\n\nviewEntryIcon : Commands.Entry -> Html Msg\nviewEntryIcon entry =\n case entry.isDir of\n True ->\n span [ class \"fas fa-lg fa-folder text-xs-right file-list-icon\" ] []\n\n False ->\n span [ class \"far fa-lg fa-file text-xs-right file-list-icon\" ] []\n\n\nviewDeletedEntry : Model -> Commands.Entry -> Table.Row Msg\nviewDeletedEntry model entry =\n Table.tr []\n [ Table.td\n []\n [ viewEntryIcon entry ]\n , Table.td\n []\n [ text entry.path ]\n , Table.td\n []\n [ text <| Util.formatLastModified model.zone entry.lastModified ]\n , Table.td\n []\n [ text <| Filesize.format entry.size ]\n , Table.td\n []\n [ Button.button\n [ Button.outlineSuccess\n , Button.attrs\n [ onClick <| UndeleteClicked entry.path\n , disabled (not (List.member \"fs.edit\" model.rights))\n ]\n ]\n [ text \"Undelete\" ]\n ]\n ]\n\n\nviewDeletedList : Model -> List Commands.Entry -> Html Msg\nviewDeletedList model entries =\n let\n filteredEntries =\n filterEntries model.filter entries\n in\n Table.table\n { options =\n [ Table.hover\n , Table.attr (class \"borderless-table\")\n ]\n , thead =\n Table.thead []\n [ Table.tr []\n [ Table.th [ Table.cellAttr (style \"width\" \"5%\") ] []\n , Table.th [ Table.cellAttr (style \"width\" \"55%\") ] [ text \"Name\" ]\n , Table.th [ Table.cellAttr (style \"width\" \"20%\") ] [ text \"Deleted at\" ]\n , Table.th [ Table.cellAttr (style \"width\" \"15%\") ] [ text \"Size\" ]\n , Table.th [ Table.cellAttr (style \"width\" \"5%\") ] []\n ]\n ]\n , tbody =\n Table.tbody []\n (List.map\n (viewDeletedEntry model)\n filteredEntries\n )\n }\n\n\nmaybeViewDeletedList : Model -> List Commands.Entry -> Html Msg\nmaybeViewDeletedList model entries =\n if List.length entries > 0 then\n viewDeletedList model entries\n\n else\n Grid.row []\n [ Grid.col [ Col.xs12, Col.textAlign Text.alignXsCenter ]\n [ span [ class \"text-muted\" ]\n (if String.length model.filter == 0 then\n [ text \" The \"\n , span [ class \"fas fa-md fa-trash-alt\" ] []\n , text \" is empty. If you delete something, it will appear here.\"\n ]\n\n else\n [ text \" Search did not find anything. Remove the query to go back. \"\n ]\n )\n ]\n ]\n\n\nviewDeletedContainer : Model -> List Commands.Entry -> Html Msg\nviewDeletedContainer model entries =\n Grid.row []\n [ Grid.col [ Col.lg1, Col.attrs [ class \"d-none d-lg-block\" ] ] []\n , Grid.col [ Col.lg10, Col.md12 ]\n [ h4 [ class \"text-muted text-center\" ] [ text \"Deleted files\" ]\n , br [] []\n , Util.viewAlert AlertMsg model.alert\n , maybeViewDeletedList model entries\n , br [] []\n ]\n , Grid.col [ Col.lg1, Col.attrs [ class \"d-none d-lg-block\" ] ] []\n ]\n\n\nview : Model -> Html Msg\nview model =\n case model.state of\n Loading ->\n text \"Still loading\"\n\n Failure err ->\n text (\"Failed to load log: \" ++ err)\n\n Success entries ->\n Grid.row []\n [ Grid.col\n [ Col.lg12 ]\n [ Grid.row [ Row.attrs [ id \"main-header-row\" ] ]\n [ Grid.col [ Col.xl9 ] []\n , Grid.col [ Col.xl3 ] [ Lazy.lazy viewSearchBox model ]\n ]\n , Grid.row [ Row.attrs [ id \"main-content-row\" ] ]\n [ Grid.col\n [ Col.xl10 ]\n [ viewDeletedContainer model entries ]\n ]\n ]\n ]\n\n\n\n-- SUBSCRIPTIONS:\n\n\nsubscriptions : Model -> Sub Msg\nsubscriptions model =\n Sub.batch\n [ Scroll.scrollOrResize OnScroll\n , Alert.subscriptions model.alert.vis AlertMsg\n ]\n" }, { "path": "gateway/elm/src/Routes/Diff.elm", "content": "module Routes.Diff exposing\n ( Model\n , Msg\n , newModel\n , reload\n , subscriptions\n , update\n , updateUrl\n , view\n )\n\nimport Bootstrap.Button as Button\nimport Bootstrap.Grid as Grid\nimport Bootstrap.Grid.Col as Col\nimport Bootstrap.Grid.Row as Row\nimport Browser.Navigation as Nav\nimport Commands\nimport Html exposing (..)\nimport Html.Attributes exposing (..)\nimport Html.Events exposing (..)\nimport Http\nimport Time\nimport Url\nimport Url.Parser exposing ((), parse, s, string)\nimport Util\n\n\n\n-- MODEL:\n\n\ntype State\n = Loading\n | Finished (Result String Commands.Diff)\n\n\ntype alias Model =\n { key : Nav.Key\n , url : Url.Url\n , zone : Time.Zone\n , state : State\n }\n\n\nnewModel : Nav.Key -> Url.Url -> Time.Zone -> Model\nnewModel key url zone =\n { key = key\n , url = url\n , zone = zone\n , state = Loading\n }\n\n\nupdateUrl : Model -> Url.Url -> Model\nupdateUrl model url =\n { model | url = url }\n\n\nnameFromUrl : Url.Url -> String\nnameFromUrl url =\n Maybe.withDefault \"\"\n (parse (s \"diff\" string) url)\n\n\nreload : Model -> Url.Url -> Cmd Msg\nreload model url =\n let\n remoteName =\n nameFromUrl url\n in\n if String.length remoteName > 0 then\n Commands.doRemoteDiff GotResponse remoteName\n\n else\n Cmd.none\n\n\n\n-- MESSAGES:\n\n\ntype Msg\n = GotResponse (Result Http.Error Commands.Diff)\n | BackClicked\n\n\n\n-- UPDATE:\n\n\nupdate : Msg -> Model -> ( Model, Cmd Msg )\nupdate msg model =\n case msg of\n GotResponse result ->\n case result of\n Ok diff ->\n ( { model | state = Finished (Ok diff) }, Cmd.none )\n\n Err err ->\n ( { model | state = Finished (Err (Util.httpErrorToString err)) }, Cmd.none )\n\n BackClicked ->\n ( model, Nav.back model.key 1 )\n\n\n\n-- VIEW:\n\n\nviewLine : String -> Html Msg\nviewLine line =\n span [] [ text line, br [] [] ]\n\n\nviewSingle : List Commands.Entry -> Html Msg -> Html Msg\nviewSingle entries header =\n if List.length entries > 0 then\n span []\n [ header\n , span [] (List.map (\\e -> viewLine <| \" \" ++ e.path) entries)\n , br [] []\n ]\n\n else\n text \"\"\n\n\nviewPairs : List Commands.DiffPair -> Html Msg -> Html Msg\nviewPairs entries header =\n if List.length entries > 0 then\n span []\n [ header\n , span [] (List.map (\\p -> viewLine (\" \" ++ p.src.path ++ \" ↔ \" ++ p.dst.path)) entries)\n , br [] []\n ]\n\n else\n text \"\"\n\n\nviewHeading : String -> String -> Html Msg\nviewHeading className message =\n h5 [ class className ] [ text message ]\n\n\nviewDiff : Model -> Commands.Diff -> Html Msg\nviewDiff model diff =\n let\n nChanges =\n Commands.diffChangeCount diff\n in\n case nChanges of\n 0 ->\n text \"There are no differences!\"\n\n n ->\n div []\n [ viewSingle diff.added (viewHeading \"text-success\" \"Added\")\n , viewSingle diff.removed (viewHeading \"text-warning\" \"Removed\")\n , viewSingle diff.ignored (viewHeading \"text-muted\" \"Ignored\")\n -- , viewSingle diff.missing (viewHeading \"text-secondary\" \"Missing\")\n , viewPairs diff.moved (viewHeading \"text-primary\" \"Moved\")\n , viewPairs diff.merged (viewHeading \"text-info\" \"Merged\")\n , viewPairs diff.conflict (viewHeading \"text-danger\" \"Conflicts\")\n , br [] []\n , br [] []\n , text (String.fromInt n ++ \" changes in total\")\n ]\n\n\nviewDiffContainer : Model -> Result String Commands.Diff -> Html Msg\nviewDiffContainer model result =\n Grid.row []\n [ Grid.col [ Col.lg2, Col.attrs [ class \"d-none d-lg-block\" ] ] []\n , Grid.col [ Col.lg8, Col.md12 ]\n [ h4 [ class \"text-center\" ]\n [ span [ class \"text-muted\" ] [ text \"Difference to »\" ]\n , text (nameFromUrl model.url)\n , span [ class \"text-muted\" ] [ text \"«\" ]\n , Button.button\n [ Button.roleLink\n , Button.attrs [ onClick BackClicked ]\n ]\n [ span [ class \"font-weight-light\" ] [ text \"(go back)\" ] ]\n ]\n , br [] []\n , case result of\n Ok diff ->\n viewDiff model diff\n\n Err err ->\n text err\n , br [] []\n ]\n , Grid.col [ Col.lg2, Col.attrs [ class \"d-none d-lg-block\" ] ] []\n ]\n\n\nview : Model -> Html Msg\nview model =\n case model.state of\n Loading ->\n text \"Still loading\"\n\n Finished result ->\n Grid.row []\n [ Grid.col\n [ Col.lg12 ]\n [ Grid.row [ Row.attrs [ id \"main-content-row\" ] ]\n [ Grid.col\n [ Col.xl10 ]\n [ viewDiffContainer model result ]\n ]\n ]\n ]\n\n\n\n-- SUBSCRIPTIONS:\n\n\nsubscriptions : Model -> Sub Msg\nsubscriptions model =\n Sub.none\n" }, { "path": "gateway/elm/src/Routes/Ls.elm", "content": "module Routes.Ls exposing\n ( Model\n , Msg\n , buildModals\n , changeTimeZone\n , changeUrl\n , doListQueryFromUrl\n , newModel\n , subscriptions\n , update\n , view\n )\n\nimport Bootstrap.Alert as Alert\nimport Bootstrap.Breadcrumb as Breadcrumb\nimport Bootstrap.Button as Button\nimport Bootstrap.ButtonGroup as ButtonGroup\nimport Bootstrap.Dropdown as Dropdown\nimport Bootstrap.Form.Input as Input\nimport Bootstrap.Form.InputGroup as InputGroup\nimport Bootstrap.Grid as Grid\nimport Bootstrap.Grid.Col as Col\nimport Bootstrap.Grid.Row as Row\nimport Bootstrap.ListGroup as ListGroup\nimport Bootstrap.Table as Table\nimport Bootstrap.Text as Text\nimport Browser.Navigation as Nav\nimport Commands\nimport Filesize\nimport Html exposing (..)\nimport Html.Attributes exposing (..)\nimport Html.Events exposing (..)\nimport Html.Lazy as Lazy\nimport Http\nimport Modals.History as History\nimport Modals.Mkdir as Mkdir\nimport Modals.MoveCopy as MoveCopy\nimport Modals.Remove as Remove\nimport Modals.Rename as Rename\nimport Modals.Share as Share\nimport Modals.Upload as Upload\nimport Set\nimport Time\nimport Url\nimport Url.Builder as UrlBuilder\nimport Url.Parser as UrlParser\nimport Url.Parser.Query as Query\nimport Util\n\n\n\n-- MODEL\n\n\ntype alias ActualModel =\n { entries : List Commands.Entry\n , checked : Set.Set String\n , isFiltered : Bool\n , self : Commands.Entry\n , sortState : ( SortDirection, SortKey )\n }\n\n\ntype State\n = Failure\n | Loading\n | Success ActualModel\n\n\ntype alias Model =\n { key : Nav.Key\n , url : Url.Url\n , zone : Time.Zone\n , state : State\n , alert : Alert.Visibility\n , currError : String\n , rights : List String\n\n -- Sub models (for modals and dialogs):\n , historyState : History.Model\n , renameState : Rename.Model\n , moveState : MoveCopy.Model\n , copyState : MoveCopy.Model\n , uploadState : Upload.Model\n , mkdirState : Mkdir.Model\n , removeState : Remove.Model\n , shareState : Share.Model\n }\n\n\nnewModel : Nav.Key -> Url.Url -> List String -> Model\nnewModel key url rights =\n { key = key\n , url = url\n , zone = Time.utc\n , state = Loading\n , alert = Alert.closed\n , rights = rights\n , currError = \"\"\n , historyState = History.newModel rights\n , renameState = Rename.newModel\n , moveState = MoveCopy.newMoveModel\n , copyState = MoveCopy.newCopyModel\n , uploadState = Upload.newModel\n , mkdirState = Mkdir.newModel\n , removeState = Remove.newModel\n , shareState = Share.newModel\n }\n\n\nchangeUrl : Url.Url -> Model -> Model\nchangeUrl url model =\n { model | url = url }\n\n\nchangeTimeZone : Time.Zone -> Model -> Model\nchangeTimeZone zone model =\n { model | zone = zone }\n\n\nnSelectedItems : Model -> Int\nnSelectedItems model =\n case model.state of\n Success actualModel ->\n Set.filter (\\e -> String.isEmpty e |> not) actualModel.checked |> Set.size\n\n _ ->\n 0\n\n\nselectedPaths : Model -> List String\nselectedPaths model =\n case model.state of\n Success actualModel ->\n Set.filter (\\e -> String.isEmpty e |> not) actualModel.checked |> Set.toList\n\n _ ->\n []\n\n\ncurrIsFile : Model -> Bool\ncurrIsFile model =\n case model.state of\n Success actualModel ->\n not actualModel.self.isDir\n\n _ ->\n False\n\n\ncurrRoot : Model -> Maybe String\ncurrRoot model =\n case model.state of\n Success actualModel ->\n Just actualModel.self.path\n\n _ ->\n Nothing\n\n\ncurrTotalSize : Model -> Int\ncurrTotalSize model =\n case model.state of\n Success actualModel ->\n actualModel.self.size\n\n _ ->\n 0\n\n\ncurrSelectedSize : Model -> Int\ncurrSelectedSize model =\n case model.state of\n Success actualModel ->\n let\n entryToSizeIfSelected =\n \\e ->\n if Set.member e.path actualModel.checked then\n e.size\n\n else\n 0\n in\n List.foldl (+) 0 (List.map entryToSizeIfSelected actualModel.entries)\n\n _ ->\n 0\n\n\nexistsInCurr : Model -> String -> Bool\nexistsInCurr model name =\n case model.state of\n Success actualModel ->\n case actualModel.isFiltered of\n True ->\n False\n\n False ->\n List.any (\\e -> name == Util.basename e.path) actualModel.entries\n\n _ ->\n False\n\n\n\n-- MESSAGES\n\n\ntype SortKey\n = None\n | Name\n | ModTime\n | Pin\n | Size\n\n\ntype SortDirection\n = Ascending\n | Descending\n\n\ntype Msg\n = GotResponse (Result Http.Error Commands.ListResponse)\n | GotPinResponse (Result Http.Error String)\n | CheckboxTick String Bool\n | CheckboxTickAll Bool\n | ActionDropdownMsg Commands.Entry Dropdown.State\n | RowClicked Commands.Entry\n | RemoveClicked Commands.Entry\n | HistoryClicked Commands.Entry\n | RemoveResponse (Result Http.Error String)\n | SortBy SortDirection SortKey\n | AlertMsg Alert.Visibility\n | SearchInput String\n | PinClicked String Bool\n -- Sub messages:\n | HistoryMsg History.Msg\n | RenameMsg Rename.Msg\n | MoveMsg MoveCopy.Msg\n | CopyMsg MoveCopy.Msg\n -- Modal sub messages:\n | UploadMsg Upload.Msg\n | MkdirMsg Mkdir.Msg\n | RemoveMsg Remove.Msg\n | ShareMsg Share.Msg\n\n\n\n-- UPDATE\n\n\nfixDropdownState : Commands.Entry -> Dropdown.State -> Commands.Entry -> Commands.Entry\nfixDropdownState refEntry state entry =\n if entry.path == refEntry.path then\n { entry | dropdown = state }\n\n else\n entry\n\n\nsortBy : ActualModel -> SortDirection -> SortKey -> ActualModel\nsortBy model direction key =\n case direction of\n Ascending ->\n { model\n | entries = sortByAscending model key\n , sortState = ( Ascending, key )\n }\n\n Descending ->\n { model\n | entries = List.reverse (sortByAscending model key)\n , sortState = ( Descending, key )\n }\n\n\nentryPinToSortKey : Commands.Entry -> Int\nentryPinToSortKey entry =\n case ( entry.isPinned, entry.isExplicit ) of\n ( True, True ) ->\n 2\n\n ( True, False ) ->\n 1\n\n _ ->\n 0\n\n\nsortByAscending : ActualModel -> SortKey -> List Commands.Entry\nsortByAscending model key =\n case key of\n Name ->\n List.sortBy (\\e -> String.toLower (Util.basename e.path)) model.entries\n\n ModTime ->\n List.sortBy (\\e -> Time.posixToMillis e.lastModified) model.entries\n\n Pin ->\n List.sortBy (\\e -> entryPinToSortKey e) model.entries\n\n Size ->\n List.sortBy .size model.entries\n\n None ->\n model.entries\n\n\nupdateCheckboxTickActual : String -> Bool -> ActualModel -> ActualModel\nupdateCheckboxTickActual path isChecked model =\n case isChecked of\n True ->\n let\n updatedSet =\n Set.insert path model.checked\n in\n { model\n | checked =\n if Set.size updatedSet == List.length model.entries then\n Set.insert \"\" updatedSet\n\n else\n updatedSet\n }\n\n False ->\n { model\n | checked =\n Set.remove \"\" <| Set.remove path model.checked\n }\n\n\nupdateCheckboxTick : String -> Bool -> Model -> Model\nupdateCheckboxTick path isChecked model =\n case model.state of\n Success actualModel ->\n { model | state = Success (updateCheckboxTickActual path isChecked actualModel) }\n\n _ ->\n model\n\n\nupdateCheckboxTickAllActual : Bool -> ActualModel -> ActualModel\nupdateCheckboxTickAllActual isChecked model =\n case isChecked of\n True ->\n { model | checked = Set.fromList (List.map (\\e -> e.path) model.entries ++ [ \"\" ]) }\n\n False ->\n { model | checked = Set.empty }\n\n\nupdateCheckboxTickAll : Bool -> Model -> Model\nupdateCheckboxTickAll isChecked model =\n case model.state of\n Success actualModel ->\n { model | state = Success (updateCheckboxTickAllActual isChecked actualModel) }\n\n _ ->\n model\n\n\nsetDropdownState : Model -> Commands.Entry -> Dropdown.State -> Model\nsetDropdownState model entry state =\n case model.state of\n Success actualModel ->\n { model\n | state =\n Success\n { actualModel\n | entries = List.map (fixDropdownState entry state) actualModel.entries\n }\n }\n\n _ ->\n model\n\n\nupdate : Msg -> Model -> ( Model, Cmd Msg )\nupdate msg model =\n case msg of\n ActionDropdownMsg entry state ->\n ( setDropdownState model entry state, Cmd.none )\n\n RowClicked entry ->\n ( model, Nav.pushUrl model.key (\"/view\" ++ Util.urlEncodePath entry.path) )\n\n RemoveClicked entry ->\n ( setDropdownState model entry Dropdown.initialState\n , Commands.doRemove RemoveResponse [ entry.path ]\n )\n\n SearchInput query ->\n ( model\n -- Save the filter query in the URL itself.\n -- This way the query can be shared amongst users via link.\n , Nav.pushUrl model.key <|\n model.url.path\n ++ (if String.length query == 0 then\n \"\"\n\n else\n UrlBuilder.toQuery\n [ UrlBuilder.string \"filter\" query\n ]\n )\n )\n\n HistoryClicked entry ->\n ( setDropdownState model entry Dropdown.initialState\n , Cmd.map HistoryMsg (History.show entry.path)\n )\n\n SortBy direction key ->\n case model.state of\n Success actualModel ->\n ( { model | state = Success (sortBy actualModel direction key) }, Cmd.none )\n\n _ ->\n ( model, Cmd.none )\n\n RemoveResponse result ->\n case result of\n Ok _ ->\n ( model, Cmd.none )\n\n Err err ->\n ( { model\n | currError = Util.httpErrorToString err\n , alert = Alert.shown\n }\n , Cmd.none\n )\n\n GotResponse result ->\n case result of\n Ok response ->\n -- New list model means also new checked entries.\n ( { model\n | state =\n Success <|\n { entries = response.entries\n , isFiltered = response.isFiltered\n , checked =\n if response.self.isDir then\n Set.empty\n\n else\n Set.singleton response.self.path\n , self = response.self\n , sortState = ( Ascending, None )\n }\n }\n , Cmd.none\n )\n\n Err _ ->\n ( { model | state = Failure }, Cmd.none )\n\n GotPinResponse result ->\n case result of\n Ok _ ->\n ( model, Cmd.none )\n\n -- TODO: Error handling?\n Err _ ->\n ( model, Cmd.none )\n\n CheckboxTick path isChecked ->\n ( updateCheckboxTick path isChecked model, Cmd.none )\n\n CheckboxTickAll isChecked ->\n ( updateCheckboxTickAll isChecked model, Cmd.none )\n\n PinClicked path shouldBePinned ->\n if shouldBePinned then\n ( model, Commands.doPin GotPinResponse path \"curr\" )\n\n else\n ( model, Commands.doUnpin GotPinResponse path \"curr\" )\n\n AlertMsg state ->\n ( { model | alert = state }, Cmd.none )\n\n HistoryMsg subMsg ->\n let\n ( newSubModel, newSubCmd ) =\n History.update subMsg model.historyState\n in\n ( { model | historyState = newSubModel }, Cmd.map HistoryMsg newSubCmd )\n\n RenameMsg subMsg ->\n let\n ( newSubModel, newSubCmd ) =\n Rename.update subMsg model.renameState\n in\n ( { model | renameState = newSubModel }, Cmd.map RenameMsg newSubCmd )\n\n MoveMsg subMsg ->\n let\n ( newSubModel, newSubCmd ) =\n MoveCopy.update subMsg model.moveState\n in\n ( { model | moveState = newSubModel }, Cmd.map MoveMsg newSubCmd )\n\n CopyMsg subMsg ->\n let\n ( newSubModel, newSubCmd ) =\n MoveCopy.update subMsg model.copyState\n in\n ( { model | copyState = newSubModel }, Cmd.map CopyMsg newSubCmd )\n\n UploadMsg subMsg ->\n let\n ( newSubModel, newSubCmd ) =\n Upload.update subMsg model.uploadState\n in\n ( { model | uploadState = newSubModel }, Cmd.map UploadMsg newSubCmd )\n\n MkdirMsg subMsg ->\n let\n ( newSubModel, newSubCmd ) =\n Mkdir.update subMsg model.mkdirState\n in\n ( { model | mkdirState = newSubModel }, Cmd.map MkdirMsg newSubCmd )\n\n RemoveMsg subMsg ->\n let\n ( newSubModel, newSubCmd ) =\n Remove.update subMsg model.removeState\n in\n ( { model | removeState = newSubModel }, Cmd.map RemoveMsg newSubCmd )\n\n ShareMsg subMsg ->\n let\n ( newSubModel, newSubCmd ) =\n Share.update subMsg model.shareState\n in\n ( { model | shareState = newSubModel }, Cmd.map ShareMsg newSubCmd )\n\n\n\n-- VIEW\n\n\nshowAlert : Model -> Html Msg\nshowAlert model =\n Alert.config\n |> Alert.dismissable AlertMsg\n |> Alert.danger\n |> Alert.children\n [ Alert.h4 [] [ text \"Oh, something went wrong! :(\" ]\n , text (\"The exact error was: \" ++ model.currError)\n ]\n |> Alert.view model.alert\n\n\nviewMetaRow : String -> Html msg -> Html msg\nviewMetaRow key value =\n Grid.row []\n [ Grid.col [ Col.xs4, Col.textAlign Text.alignXsLeft ] [ span [ class \"text-muted\" ] [ text key ] ]\n , Grid.col [ Col.xs8, Col.textAlign Text.alignXsRight ] [ value ]\n ]\n\n\nviewDownloadButton : Model -> ActualModel -> Url.Url -> Html msg\nviewDownloadButton model actModel url =\n Button.linkButton\n [ Button.outlinePrimary\n , Button.attrs\n (if mayDownload model then\n [ href\n (Util.urlPrefixToString url\n ++ \"get\"\n ++ Util.urlEncodePath actModel.self.path\n ++ \"?direct=yes\"\n )\n ]\n\n else\n [ class \"text-muted\", style \"opacity\" \"0.1\" ]\n )\n ]\n [ span [ class \"fas fa-download\" ] [], text \" Download\" ]\n\n\nviewViewButton : Model -> ActualModel -> Url.Url -> Html msg\nviewViewButton model actModel url =\n Button.linkButton\n [ Button.outlinePrimary\n , Button.attrs\n (if mayDownload model then\n [ href\n (Util.urlPrefixToString url\n ++ \"get\"\n ++ Util.urlEncodePath actModel.self.path\n )\n ]\n\n else\n [ class \"text-muted\", style \"opacity\" \"0.1\" ]\n )\n ]\n [ span [ class \"fas fa-eye\" ] [], text \" View\" ]\n\n\nviewPinIcon : Bool -> Bool -> Html msg\nviewPinIcon isPinned isExplicit =\n case ( isPinned, isExplicit ) of\n ( True, True ) ->\n span\n [ class \"fa fa-map-marker\", class \"text-success\" ]\n []\n\n ( True, False ) ->\n span\n [ class \"fa fa-map-marker-alt\", class \"text-warning\" ]\n []\n\n _ ->\n span\n [ class \"fa fa-times\", class \"text-danger\" ]\n []\n\n\nviewPinButton : Model -> Commands.Entry -> Html Msg\nviewPinButton model entry =\n Button.button\n [ Button.roleLink\n , Button.attrs\n [ disabled (not (List.member \"fs.edit\" model.rights))\n , onClick (PinClicked entry.path (not entry.isPinned))\n ]\n ]\n [ viewPinIcon entry.isPinned entry.isExplicit ]\n\n\nviewSingleEntry : Model -> ActualModel -> Time.Zone -> Html Msg\nviewSingleEntry model actualModel zone =\n Grid.row []\n [ Grid.col [ Col.xs2 ] []\n , Grid.col [ Col.xs8, Col.textAlign Text.alignXsCenter ]\n [ ListGroup.ul\n [ ListGroup.li []\n [ viewMetaRow \"Path\" (text <| actualModel.self.path)\n ]\n , ListGroup.li []\n [ viewMetaRow \"Size\" (text <| Filesize.format actualModel.self.size)\n ]\n , ListGroup.li []\n [ viewMetaRow \"Owner\" (text <| actualModel.self.user)\n ]\n , ListGroup.li []\n [ viewMetaRow \"Last Modified\" (text <| Util.formatLastModified zone actualModel.self.lastModified)\n ]\n , ListGroup.li []\n [ viewMetaRow \"Pinned\"\n (viewPinButton model actualModel.self)\n ]\n , ListGroup.li [ ListGroup.light ]\n [ viewDownloadButton model actualModel model.url\n , text \" \"\n , viewViewButton model actualModel model.url\n ]\n ]\n ]\n , Grid.col [ Col.xs2 ] []\n ]\n\n\nviewList : Model -> Time.Zone -> Html Msg\nviewList model zone =\n case model.state of\n Failure ->\n div [] [ text \"Sorry, something did not work out as expected.\" ]\n\n Loading ->\n text \"Loading...\"\n\n Success actualModel ->\n case actualModel.self.isDir of\n True ->\n div []\n [ showAlert model\n , Lazy.lazy3 entriesToHtml model zone actualModel\n ]\n\n False ->\n div []\n [ showAlert model\n , Lazy.lazy3 viewSingleEntry model actualModel zone\n ]\n\n\nbuildBreadcrumbs : List String -> List String -> List (Breadcrumb.Item msg)\nbuildBreadcrumbs names previous =\n let\n displayName =\n \\n ->\n if String.length n <= 0 then\n \"Home\"\n\n else\n n\n in\n case names of\n [] ->\n -- Recursion stop.\n []\n\n [ name ] ->\n -- Final element in the breadcrumbs.\n -- Already selected therefore.\n [ Breadcrumb.item []\n [ text (displayName name)\n ]\n ]\n\n name :: rest ->\n -- Some intermediate element.\n Breadcrumb.item []\n [ a [ href (\"/view/\" ++ String.join \"/\" (name :: previous)) ]\n [ text (displayName name) ]\n ]\n :: buildBreadcrumbs rest (previous ++ [ name ])\n\n\nviewBreadcrumbs : Model -> Html msg\nviewBreadcrumbs model =\n div [ id \"breadcrumbs-box\" ]\n [ Breadcrumb.container\n (buildBreadcrumbs\n (\"\" :: (Util.urlToPath model.url |> Util.splitPath))\n []\n )\n ]\n\n\nviewEntryIcon : Commands.Entry -> Html Msg\nviewEntryIcon entry =\n case entry.isDir of\n True ->\n span [ class \"fas fa-lg fa-folder text-xs-right file-list-icon\" ] []\n\n False ->\n span [ class \"far fa-lg fa-file text-xs-right file-list-icon\" ] []\n\n\nmakeCheckbox : Bool -> (Bool -> Msg) -> Html Msg\nmakeCheckbox isChecked msg =\n div [ class \"checkbox\" ]\n [ label []\n [ input [ type_ \"checkbox\", onCheck msg, checked isChecked ] []\n , span [ class \"cr\" ] [ i [ class \"cr-icon fas fa-lg fa-check\" ] [] ]\n ]\n ]\n\n\nreadCheckedState : ActualModel -> String -> Bool\nreadCheckedState model path =\n Set.member path model.checked\n\n\nformatPath : ActualModel -> Commands.Entry -> String\nformatPath model entry =\n case model.isFiltered of\n True ->\n String.join \"/\" (Util.splitPath entry.path)\n\n False ->\n Util.basename entry.path\n\n\nmayDownload : Model -> Bool\nmayDownload model =\n List.member \"fs.download\" model.rights\n\n\nmayEdit : Model -> Bool\nmayEdit model =\n List.member \"fs.edit\" model.rights\n\n\nbuildActionDropdown : Model -> ActualModel -> Commands.Entry -> Html Msg\nbuildActionDropdown model actModel entry =\n Dropdown.dropdown\n entry.dropdown\n { options = []\n , toggleMsg = ActionDropdownMsg entry\n , toggleButton =\n Dropdown.toggle\n [ Button.roleLink ]\n [ span [ class \"fas fa-ellipsis-h\" ] [] ]\n , items =\n [ Dropdown.buttonItem\n [ onClick (HistoryClicked entry) ]\n [ span [ class \"fa fa-md fa-history\" ] []\n , text \" History\"\n ]\n , Dropdown.divider\n , Dropdown.anchorItem\n [ href\n (\"/get\"\n ++ Util.urlEncodePath\n (Util.joinPath [ actModel.self.path, Util.basename entry.path ])\n ++ \"?direct=yes\"\n )\n , onClick (ActionDropdownMsg entry Dropdown.initialState)\n , disabled (not (mayDownload model))\n ]\n [ span [ class \"fa fa-md fa-file-download\" ] []\n , text \" Download\"\n ]\n , Dropdown.anchorItem\n [ href\n (\"/get\"\n ++ Util.urlEncodePath\n (Util.joinPath [ actModel.self.path, Util.basename entry.path ])\n )\n , onClick (ActionDropdownMsg entry Dropdown.initialState)\n , disabled (not (mayDownload model))\n ]\n [ span [ class \"fa fa-md fa-eye\" ] []\n , text \" View\"\n ]\n , Dropdown.anchorItem\n [ onClick (ShareMsg <| Share.show [ entry.path ])\n ]\n [ span [ class \"fa fa-md fa-share-alt\" ] []\n , text \" Share\"\n ]\n , Dropdown.divider\n , Dropdown.buttonItem\n [ onClick (RemoveClicked entry)\n , disabled (not (mayEdit model))\n ]\n [ span [ class \"fa fa-md fa-trash\" ] []\n , text \" Delete\"\n ]\n , Dropdown.divider\n , Dropdown.buttonItem\n [ onClick (RenameMsg (Rename.show entry.path))\n , disabled (not (mayEdit model))\n ]\n [ span [ class \"fa fa-md fa-file-signature\" ] []\n , text \" Rename\"\n ]\n , Dropdown.buttonItem\n [ onClick (MoveMsg (MoveCopy.show entry.path))\n , disabled (not (mayEdit model))\n ]\n [ span [ class \"fa fa-md fa-arrow-right\" ] []\n , text \" Move\"\n ]\n , Dropdown.buttonItem\n [ onClick (CopyMsg (MoveCopy.show entry.path))\n , disabled (not (mayEdit model))\n ]\n [ span [ class \"fa fa-md fa-copy\" ] []\n , text \" Copy\"\n ]\n ]\n }\n\n\nentryToHtml : Model -> ActualModel -> Time.Zone -> Commands.Entry -> Table.Row Msg\nentryToHtml model actModel zone e =\n Table.tr\n []\n [ Table.td []\n [ makeCheckbox (readCheckedState actModel e.path) (CheckboxTick e.path)\n ]\n , Table.td\n [ Table.cellAttr (class \"icon-column\"), Table.cellAttr (onClick (RowClicked e)) ]\n [ viewEntryIcon e ]\n , Table.td\n [ Table.cellAttr (onClick (RowClicked e)) ]\n [ a [ \"/view\" ++ e.path |> href ] [ text (formatPath actModel e) ]\n ]\n , Table.td\n [ Table.cellAttr (onClick (RowClicked e)) ]\n [ Util.formatLastModifiedOwner zone e.lastModified e.user\n ]\n , Table.td\n [ Table.cellAttr (onClick (RowClicked e)) ]\n [ text (Filesize.format e.size)\n ]\n , Table.td\n []\n [ viewPinButton model e\n ]\n , Table.td\n []\n [ buildActionDropdown model actModel e\n ]\n ]\n\n\nbuildSortControl : String -> ActualModel -> SortKey -> Html Msg\nbuildSortControl name model key =\n let\n ascClass =\n if ( Ascending, key ) == model.sortState then\n \"sort-button-selected\"\n\n else\n \"\"\n\n descClass =\n if ( Descending, key ) == model.sortState then\n \"sort-button-selected\"\n\n else\n \"\"\n in\n span [ class \"sort-button-container text-muted\" ]\n [ span [] [ text (name ++ \" \") ]\n , span [ class \"sort-button\" ]\n [ Button.linkButton\n [ Button.small\n , Button.attrs [ onClick (SortBy Ascending key), class \"sort-button\" ]\n ]\n [ span\n [ class \"fas fa-xs fa-arrow-up\", class ascClass ]\n []\n ]\n , Button.linkButton\n [ Button.small\n , Button.attrs [ onClick (SortBy Descending key), class \"sort-button\" ]\n ]\n [ span [ class \"fas fa-xs fa-arrow-down\", class descClass ] [] ]\n ]\n ]\n\n\nentriesToHtml : Model -> Time.Zone -> ActualModel -> Html Msg\nentriesToHtml model zone actModel =\n Table.table\n { options = [ Table.hover ]\n , thead =\n Table.simpleThead\n [ Table.th [ Table.cellAttr (style \"width\" \"5%\") ]\n [ makeCheckbox (readCheckedState actModel \"\") CheckboxTickAll\n ]\n , Table.th [ Table.cellAttr (style \"width\" \"5%\") ]\n [ text \"\" ]\n , Table.th [ Table.cellAttr (style \"width\" \"37.5%\") ]\n [ buildSortControl \"Name\" actModel Name ]\n , Table.th [ Table.cellAttr (style \"width\" \"27.5%\") ]\n [ buildSortControl \"Modified\" actModel ModTime ]\n , Table.th [ Table.cellAttr (style \"width\" \"7.5%\") ]\n [ buildSortControl \"Size\" actModel Size ]\n , Table.th [ Table.cellAttr (style \"width\" \"10%\") ]\n [ buildSortControl \"Pin\" actModel Pin ]\n , Table.th [ Table.cellAttr (style \"width\" \"5%\") ]\n [ text \"\" ]\n ]\n , tbody =\n Table.tbody []\n (List.map (entryToHtml model actModel zone) actModel.entries)\n }\n\n\nbuildModals : Model -> Html Msg\nbuildModals model =\n let\n paths =\n selectedPaths model\n in\n span []\n [ Html.map HistoryMsg (History.view model.historyState)\n , Html.map RenameMsg (Rename.view model.renameState)\n , Html.map MoveMsg (MoveCopy.view model.moveState)\n , Html.map CopyMsg (MoveCopy.view model.copyState)\n , Html.map MkdirMsg (Mkdir.view model.mkdirState model.url (existsInCurr model))\n , Html.map RemoveMsg (Remove.view model.removeState paths)\n , Html.map ShareMsg (Share.view model.shareState model.url)\n ]\n\n\nsearchQueryFromUrl : Url.Url -> String\nsearchQueryFromUrl url =\n Maybe.withDefault \"\"\n (UrlParser.parse\n (UrlParser.query\n (Query.map (Maybe.withDefault \"\") (Query.string \"filter\"))\n )\n { url | path = \"\" }\n )\n\n\ndoListQueryFromUrl : Url.Url -> Cmd Msg\ndoListQueryFromUrl url =\n let\n path =\n Util.urlToPath url\n\n filter =\n searchQueryFromUrl url\n in\n Commands.doListQuery GotResponse path filter\n\n\nviewSearchBox : Model -> Html Msg\nviewSearchBox model =\n InputGroup.config\n (InputGroup.text\n [ Input.placeholder \"Search\"\n , Input.attrs\n [ onInput SearchInput\n , value (searchQueryFromUrl model.url)\n ]\n ]\n )\n |> InputGroup.successors\n [ InputGroup.span [ class \"input-group-addon\" ]\n [ button [] [ span [ class \"fas fa-search fa-xs input-group-addon\" ] [] ]\n ]\n ]\n |> InputGroup.attrs [ class \"stylish-input-group input-group\" ]\n |> InputGroup.view\n\n\nbuildActionButton : Msg -> String -> String -> Bool -> Html Msg\nbuildActionButton msg iconName labelText isDisabled =\n Button.button\n [ Button.block\n , Button.small\n , Button.roleLink\n , Button.attrs [ class \"text-left\", disabled isDisabled, onClick msg ]\n ]\n [ span [ class \"fas fa-lg\", class iconName ] []\n , span [ class \"d-lg-inline d-none\" ] [ text (\" \" ++ labelText) ]\n ]\n\n\nlabelSelectedItems : Model -> Int -> Html Msg\nlabelSelectedItems model num =\n if currIsFile model then\n text \"\"\n\n else\n case num of\n 0 ->\n p []\n [ text \" Nothing selected\"\n , br [] []\n , text (Filesize.format (currTotalSize model) ++ \" in total\")\n ]\n\n 1 ->\n p []\n [ text \" 1 item\"\n , br [] []\n , text (Filesize.format (currSelectedSize model))\n ]\n\n n ->\n p []\n [ text (\" \" ++ String.fromInt n ++ \" items\")\n , br [] []\n , text (Filesize.format (currSelectedSize model))\n ]\n\n\nbuildDownloadUrl : Model -> String\nbuildDownloadUrl model =\n UrlBuilder.absolute\n (\"get\" :: (Util.splitPath <| Util.urlToPath model.url))\n (UrlBuilder.string \"direct\" \"yes\"\n :: (if nSelectedItems model > 0 then\n List.map (UrlBuilder.string \"include\") (selectedPaths model)\n\n else\n []\n )\n )\n\n\nviewSidebarDownloadButton : Model -> Html Msg\nviewSidebarDownloadButton model =\n let\n nSelected =\n nSelectedItems model\n\n disabledClass =\n if currIsFile model || not (List.member \"fs.download\" model.rights) then\n class \"disabled\"\n\n else\n class \"btn-default\"\n in\n Button.linkButton\n [ Button.block\n , Button.attrs\n [ class \"text-left btn-link download-btn\"\n , disabledClass\n , href (buildDownloadUrl model)\n ]\n ]\n [ span [ class \"fas fa-lg fa-file-download\" ] []\n , span [ id \"action-btn\", class \"d-none d-lg-inline\" ]\n [ if nSelected > 0 then\n text \" Download selected \"\n\n else\n text \" Download all\"\n ]\n ]\n\n\nneedsRight : Model -> String -> something -> something -> something\nneedsRight model right entry default =\n if List.member right model.rights then\n entry\n\n else\n default\n\n\nviewActionList : Model -> Html Msg\nviewActionList model =\n let\n nSelected =\n nSelectedItems model\n\n root =\n Maybe.withDefault \"/\" (currRoot model)\n in\n div []\n [ div [ class \"d-flex flex-lg-column flex-row\" ]\n [ p\n [ class \"text-muted\", id \"select-label\" ]\n [ labelSelectedItems model nSelected ]\n , div [ class \"d-flex flex-column\" ]\n [ Upload.buildButton\n model.uploadState\n (currIsFile model || not (List.member \"fs.download\" model.rights))\n root\n UploadMsg\n , viewSidebarDownloadButton model\n ]\n , div [ class \"d-flex flex-column\" ]\n [ buildActionButton\n (ShareMsg <| Share.show (selectedPaths model))\n \"fa-share-alt\"\n \"Share\"\n (nSelected == 0)\n , buildActionButton\n (MkdirMsg <| Mkdir.show)\n \"fa-edit\"\n \"New Folder\"\n (currIsFile model || not (List.member \"fs.edit\" model.rights))\n ]\n , div [ class \"d-flex flex-column\" ]\n [ buildActionButton\n (RemoveMsg <| Remove.show (selectedPaths model))\n \"fa-trash\"\n \"Delete\"\n (currIsFile model || nSelected == 0 || not (List.member \"fs.edit\" model.rights))\n ]\n ]\n , div []\n [ Html.map UploadMsg (Upload.viewUploadState model.uploadState)\n ]\n ]\n\n\nview : Model -> Html Msg\nview model =\n Grid.row []\n [ Grid.col\n [ Col.lg12 ]\n [ Grid.row [ Row.attrs [ id \"main-header-row\" ] ]\n [ Grid.col [ Col.xl9 ]\n [ viewBreadcrumbs model ]\n , Grid.col [ Col.xl3 ] [ Lazy.lazy viewSearchBox model ]\n ]\n , Grid.row [ Row.attrs [ id \"main-content-row\" ] ]\n [ Grid.col\n [ Col.xl10 ]\n [ viewList model model.zone ]\n , Grid.col [ Col.xl2 ] [ Lazy.lazy viewActionList model ]\n ]\n ]\n ]\n\n\n\n-- SUBSCRIPTIONS\n\n\nsubscriptions : Model -> Sub Msg\nsubscriptions model =\n case model.state of\n Success actualModel ->\n Sub.batch\n [ Alert.subscriptions model.alert AlertMsg\n , Sub.map HistoryMsg (History.subscriptions model.historyState)\n , Sub.map RenameMsg (Rename.subscriptions model.renameState)\n , Sub.map MoveMsg (MoveCopy.subscriptions model.moveState)\n , Sub.map CopyMsg (MoveCopy.subscriptions model.copyState)\n , Sub.map UploadMsg (Upload.subscriptions model.uploadState)\n , Sub.map MkdirMsg (Mkdir.subscriptions model.url model.mkdirState)\n , Sub.map RemoveMsg (Remove.subscriptions model.removeState)\n , Sub.map ShareMsg (Share.subscriptions model.shareState)\n , Sub.batch\n (List.map (\\e -> Dropdown.subscriptions e.dropdown (ActionDropdownMsg e))\n actualModel.entries\n )\n ]\n\n _ ->\n Sub.none\n" }, { "path": "gateway/elm/src/Routes/Remotes.elm", "content": "module Routes.Remotes exposing\n ( Model\n , Msg\n , buildModals\n , newModel\n , reload\n , subscriptions\n , update\n , view\n )\n\nimport Bootstrap.Alert as Alert\nimport Bootstrap.Button as Button\nimport Bootstrap.Dropdown as Dropdown\nimport Bootstrap.Grid as Grid\nimport Bootstrap.Grid.Col as Col\nimport Bootstrap.Grid.Row as Row\nimport Bootstrap.ListGroup as ListGroup\nimport Bootstrap.Table as Table\nimport Bootstrap.Text as Text\nimport Browser.Navigation as Nav\nimport Clipboard\nimport Commands\nimport Delay\nimport Dict\nimport Html exposing (..)\nimport Html.Attributes exposing (..)\nimport Html.Events exposing (..)\nimport Http\nimport Json.Encode as E\nimport Modals.RemoteAdd as RemoteAdd\nimport Modals.RemoteFolders as RemoteFolders\nimport Modals.RemoteRemove as RemoteRemove\nimport Time\nimport Tuple\nimport Url\nimport Util\n\n\n\n-- MODEL:\n\n\ntype State\n = Loading\n | Failure String\n | Success (List Commands.Remote)\n\n\ntype alias Model =\n { key : Nav.Key\n , zone : Time.Zone\n , state : State\n , self : Commands.SelfResponse\n , alert : Util.AlertState\n , remoteAddState : RemoteAdd.Model\n , remoteRemoveState : RemoteRemove.Model\n , remoteFoldersState : RemoteFolders.Model\n , actionDropdowns : Dict.Dict String Dropdown.State\n , conflictDropdowns : Dict.Dict String Dropdown.State\n , rights : List String\n }\n\n\nnewModel : Nav.Key -> Time.Zone -> List String -> Model\nnewModel key zone rights =\n { key = key\n , zone = zone\n , state = Loading\n , rights = rights\n , self = Commands.emptySelf\n , remoteAddState = RemoteAdd.newModel\n , remoteRemoveState = RemoteRemove.newModel\n , remoteFoldersState = RemoteFolders.newModel\n , actionDropdowns = Dict.empty\n , conflictDropdowns = Dict.empty\n , alert = Util.defaultAlertState\n }\n\n\n\n-- MESSAGES:\n\n\ntype Msg\n = GotRemoteListResponse (Result Http.Error (List Commands.Remote))\n | GotSyncResponse (Result Http.Error String)\n | GotSelfResponse (Result Http.Error Commands.SelfResponse)\n | GotRemoteModifyResponse (Result Http.Error String)\n | SyncClicked String\n | AutoUpdateToggled Commands.Remote Bool\n | AcceptPushToggled Commands.Remote Bool\n | ConflictStrategyToggled Commands.Remote String\n | ClipboardCopyClicked String\n -- Sub messages:\n | RemoteAddMsg RemoteAdd.Msg\n | RemoteRemoveMsg RemoteRemove.Msg\n | RemoteFolderMsg RemoteFolders.Msg\n | ActionDropdownMsg String Dropdown.State\n | ConflictDropdownMsg String Dropdown.State\n | AlertMsg Alert.Visibility\n\n\n\n-- UPDATE:\n\n\nreload : Cmd Msg\nreload =\n Cmd.batch\n [ Commands.doRemoteList GotRemoteListResponse\n , Commands.doSelfQuery GotSelfResponse\n ]\n\n\nshowAlert : Model -> Float -> Util.AlertType -> String -> ( Model, Cmd Msg )\nshowAlert model duration modalTyp message =\n let\n newAlert =\n Util.AlertState message modalTyp Alert.shown\n in\n ( { model | alert = newAlert }\n , Cmd.batch\n [ Delay.after duration Delay.Second (AlertMsg Alert.closed) ]\n )\n\n\nupdate : Msg -> Model -> ( Model, Cmd Msg )\nupdate msg model =\n case msg of\n GotRemoteListResponse result ->\n case result of\n Ok remotes ->\n ( { model | state = Success remotes }, Cmd.none )\n\n Err err ->\n ( { model | state = Failure (Util.httpErrorToString err) }, Cmd.none )\n\n GotSyncResponse result ->\n case result of\n Ok _ ->\n showAlert model 5 Util.Success \"Succesfully synchronized!\"\n\n Err err ->\n showAlert model 20 Util.Danger (\"Failed to sync: \" ++ Util.httpErrorToString err)\n\n GotRemoteModifyResponse result ->\n case result of\n Ok _ ->\n ( model, Cmd.none )\n\n Err err ->\n showAlert model 20 Util.Danger (\"Failed to set auto update: \" ++ Util.httpErrorToString err)\n\n GotSelfResponse result ->\n case result of\n Ok self ->\n ( { model | self = self }, Cmd.none )\n\n Err err ->\n showAlert model 20 Util.Danger (\"Failed to get information about ourselves: \" ++ Util.httpErrorToString err)\n\n ActionDropdownMsg name state ->\n ( { model | actionDropdowns = Dict.insert name state model.actionDropdowns }, Cmd.none )\n\n ConflictDropdownMsg name state ->\n ( { model | conflictDropdowns = Dict.insert name state model.conflictDropdowns }, Cmd.none )\n\n SyncClicked name ->\n ( model, Commands.doRemoteSync GotSyncResponse name )\n\n AutoUpdateToggled remote state ->\n ( model, Commands.doRemoteModify GotRemoteModifyResponse { remote | acceptAutoUpdates = state } )\n\n AcceptPushToggled remote state ->\n ( model, Commands.doRemoteModify GotRemoteModifyResponse { remote | acceptPush = state } )\n\n ConflictStrategyToggled remote state ->\n ( model, Commands.doRemoteModify GotRemoteModifyResponse { remote | conflictStrategy = state } )\n\n RemoteAddMsg subMsg ->\n let\n ( upModel, upCmd ) =\n RemoteAdd.update subMsg model.remoteAddState\n in\n ( { model | remoteAddState = upModel }, Cmd.map RemoteAddMsg upCmd )\n\n RemoteRemoveMsg subMsg ->\n let\n ( upModel, upCmd ) =\n RemoteRemove.update subMsg model.remoteRemoveState\n in\n ( { model | remoteRemoveState = upModel }, Cmd.map RemoteRemoveMsg upCmd )\n\n RemoteFolderMsg subMsg ->\n let\n ( upModel, upCmd ) =\n RemoteFolders.update subMsg model.remoteFoldersState\n in\n ( { model | remoteFoldersState = upModel }, Cmd.map RemoteFolderMsg upCmd )\n\n AlertMsg vis ->\n let\n newAlert =\n Util.AlertState model.alert.message model.alert.typ vis\n in\n ( { model | alert = newAlert }, Cmd.none )\n\n ClipboardCopyClicked fingerprint ->\n let\n newModelCmd =\n showAlert model 2 Util.Info \"Copied full fingerprint to clipboard.\"\n in\n ( Tuple.first newModelCmd\n , Cmd.batch\n [ Clipboard.copyToClipboard <| E.string fingerprint\n , Tuple.second newModelCmd\n ]\n )\n\n\n\n-- VIEW:\n\n\nviewAutoUpdatesIcon : Bool -> Commands.Remote -> Bool -> Html Msg\nviewAutoUpdatesIcon state remote isDisabled =\n Util.viewToggleSwitch (AutoUpdateToggled remote) \"\" state isDisabled\n\n\nviewAcceptPushToggle : Bool -> Commands.Remote -> Bool -> Html Msg\nviewAcceptPushToggle state remote isDisabled =\n Util.viewToggleSwitch (AcceptPushToggled remote) \"\" state isDisabled\n\n\nviewRemoteState : Model -> Commands.Remote -> Html Msg\nviewRemoteState model remote =\n if remote.isAuthenticated then\n if remote.isOnline then\n span [ class \"fas fa-md fa-circle text-success\" ] []\n\n else\n span [ class \"text-warning\" ]\n [ text <| Util.formatLastModified model.zone remote.lastSeen ]\n\n else\n span [ class \"text-danger\" ] [ text \"not authenticated\" ]\n\n\nviewFingerprintButton : String -> Html Msg\nviewFingerprintButton fingerprint =\n Button.button\n [ Button.roleLink\n , Button.attrs\n [ onClick <| ClipboardCopyClicked fingerprint ]\n ]\n [ viewFingerprint fingerprint ]\n\n\nviewFingerprint : String -> Html Msg\nviewFingerprint fingerprint =\n String.split \":\" fingerprint\n |> List.map (\\t -> span [ class \"text-muted\" ] [ text (String.slice 0 10 t) ])\n |> List.intersperse (span [] [ text \":\" ])\n |> span [ class \"fingerprint\" ]\n\n\nviewActionDropdown : Model -> Commands.Remote -> Html Msg\nviewActionDropdown model remote =\n Dropdown.dropdown\n (Maybe.withDefault Dropdown.initialState (Dict.get remote.name model.actionDropdowns))\n { options = [ Dropdown.alignMenuRight ]\n , toggleMsg = ActionDropdownMsg remote.name\n , toggleButton =\n Dropdown.toggle\n [ Button.roleLink ]\n [ span [ class \"fas fa-ellipsis-h\" ] [] ]\n , items =\n [ Dropdown.buttonItem\n [ onClick (SyncClicked remote.name)\n , disabled\n (not remote.isAuthenticated\n || not (List.member \"fs.edit\" model.rights)\n )\n ]\n [ span [ class \"fas fa-md fa-sync-alt\" ] [], text \" Sync\" ]\n , Dropdown.anchorItem\n [ disabled\n (not remote.isAuthenticated\n || not (List.member \"fs.view\" model.rights)\n )\n , if remote.isAuthenticated then\n href (\"/diff/\" ++ Url.percentEncode remote.name)\n\n else\n class \"text-muted\"\n ]\n [ span [ class \"fas fa-md fa-search-minus\" ] [], text \" Diff\" ]\n , Dropdown.divider\n , Dropdown.buttonItem\n [ onClick (RemoteRemoveMsg <| RemoteRemove.show remote.name)\n , disabled (not (List.member \"remotes.edit\" model.rights))\n ]\n [ span [ class \"text-danger\" ]\n [ span [ class \"fas fa-md fa-times\" ] []\n , text \" Remove\"\n ]\n ]\n ]\n }\n\n\nconflictStrategyToIconName : Model -> String -> String\nconflictStrategyToIconName model strategy =\n case strategy of\n \"\" ->\n if model.self.defaultConflictStrategy == \"\" then\n \"fa-question text-muted\"\n\n else\n conflictStrategyToIconName model model.self.defaultConflictStrategy\n\n \"ignore\" ->\n \"fa-eject\"\n\n \"marker\" ->\n \"fa-marker\"\n\n \"embrace\" ->\n \"fa-handshake\"\n\n _ ->\n \"fa-question\"\n\n\nviewConflictDropdown : Model -> Commands.Remote -> Bool -> Html Msg\nviewConflictDropdown model remote isDisabled =\n Dropdown.dropdown\n (Maybe.withDefault Dropdown.initialState (Dict.get remote.name model.conflictDropdowns))\n { options =\n [ Dropdown.alignMenuRight\n , Dropdown.attrs [ disabled isDisabled ]\n ]\n , toggleMsg = ConflictDropdownMsg remote.name\n , toggleButton =\n Dropdown.toggle\n [ Button.roleLink, Button.attrs [ disabled isDisabled ] ]\n [ span [ class \"fas\", class <| conflictStrategyToIconName model remote.conflictStrategy ] [] ]\n , items =\n [ Dropdown.buttonItem\n [ onClick (ConflictStrategyToggled remote \"ignore\")\n , disabled isDisabled\n ]\n [ span [ class \"fas fa-md fa-eject\" ] [], text \" Ignore\" ]\n , Dropdown.buttonItem\n [ onClick (ConflictStrategyToggled remote \"marker\")\n , disabled isDisabled\n ]\n [ span [ class \"fas fa-md fa-marker\" ] [], text \" Marker\" ]\n , Dropdown.buttonItem\n [ onClick (ConflictStrategyToggled remote \"embrace\")\n , disabled isDisabled\n ]\n [ span [ class \"fas fa-md fa-handshake\" ] [], text \" Embrace\" ]\n , Dropdown.buttonItem\n [ onClick (ConflictStrategyToggled remote \"\")\n , disabled isDisabled\n ]\n [ span [ class \"fas fa-md fa-eraser\" ] [], text \" Default\" ]\n ]\n }\n\n\nviewRemote : Model -> Commands.Remote -> Table.Row Msg\nviewRemote model remote =\n let\n isDisabled =\n not (List.member \"remotes.edit\" model.rights)\n in\n Table.tr []\n [ Table.td\n []\n [ span [ class \"fas fa-lg fa-user-circle text-xs-right\" ] [] ]\n , Table.td\n []\n [ text <| \" \" ++ remote.name ]\n , Table.td\n []\n [ viewRemoteState model remote ]\n , Table.td\n []\n [ span [ class \"text-muted\" ] [ viewFingerprintButton remote.fingerprint ] ]\n , Table.td\n []\n [ viewAutoUpdatesIcon remote.acceptAutoUpdates remote isDisabled ]\n , Table.td\n []\n [ viewAcceptPushToggle remote.acceptPush remote isDisabled ]\n , Table.td\n []\n [ viewConflictDropdown model remote isDisabled ]\n , Table.td\n []\n [ Button.button\n [ Button.roleLink\n , Button.attrs\n [ onClick <| RemoteFolderMsg (RemoteFolders.show remote)\n , disabled isDisabled\n ]\n ]\n [ span\n []\n [ case List.length remote.folders of\n 0 ->\n span [ class \"fas fa-xs fa-asterisk\" ] []\n\n n ->\n text <| String.fromInt n\n ]\n ]\n ]\n , Table.td\n [ Table.cellAttr (class \"text-right\") ]\n [ viewActionDropdown model remote ]\n ]\n\n\nviewRemoteList : Model -> List Commands.Remote -> Html Msg\nviewRemoteList model remotes =\n Table.table\n { options =\n [ Table.hover\n , Table.attr (class \"borderless-table\")\n ]\n , thead =\n Table.thead []\n [ Table.tr []\n [ Table.th\n [ Table.cellAttr (style \"width\" \"5%\") ]\n [ text \"\" ]\n , Table.th\n [ Table.cellAttr (style \"width\" \"20%\") ]\n [ span [ class \"text-muted remote-heading\" ] [ text \"Name\" ] ]\n , Table.th\n [ Table.cellAttr (style \"width\" \"20%\") ]\n [ span [ class \"text-muted remote-heading\" ] [ text \"Online\" ] ]\n , Table.th\n [ Table.cellAttr (style \"width\" \"30%\") ]\n [ span [ class \"text-muted remote-heading\" ] [ text \"Fingerprint\" ] ]\n , Table.th\n [ Table.cellAttr (style \"width\" \"10%\") ]\n [ span [ class \"text-muted remote-heading\" ] [ text \"Auto Update\" ] ]\n , Table.th\n [ Table.cellAttr (style \"width\" \"10%\") ]\n [ span [ class \"text-muted remote-heading\" ] [ text \"May Push\" ] ]\n , Table.th\n [ Table.cellAttr (style \"width\" \"10%\") ]\n [ span [ class \"text-muted remote-heading\" ] [ text \"Conflicts\" ] ]\n , Table.th\n [ Table.cellAttr (style \"width\" \"10%\") ]\n [ span [ class \"text-muted remote-heading\" ] [ text \"Folders\" ] ]\n , Table.th\n [ Table.cellAttr (style \"width\" \"5%\") ]\n []\n ]\n ]\n , tbody =\n Table.tbody []\n (List.map\n (viewRemote model)\n remotes\n )\n }\n\n\nviewMetaRow : String -> Html msg -> Html msg\nviewMetaRow key value =\n Grid.row []\n [ Grid.col [ Col.xs4, Col.textAlign Text.alignXsLeft ] [ span [ class \"text-muted\" ] [ text key ] ]\n , Grid.col [ Col.xs8, Col.textAlign Text.alignXsRight ] [ value ]\n ]\n\n\nviewSelf : Model -> Html Msg\nviewSelf model =\n Grid.row []\n [ Grid.col [ Col.lg2, Col.attrs [ class \"d-none d-lg-block\" ] ] []\n , Grid.col [ Col.xs12, Col.lg8, Col.textAlign Text.alignXsCenter ]\n [ ListGroup.ul\n [ ListGroup.li []\n [ viewMetaRow \"Name\" (text model.self.self.name)\n ]\n , ListGroup.li []\n [ viewMetaRow \"Fingerprint\" (viewFingerprintButton model.self.self.fingerprint)\n ]\n ]\n ]\n , Grid.col [ Col.lg2, Col.attrs [ class \"d-none d-lg-block\" ] ] []\n ]\n\n\nviewRemoteListContainer : Model -> List Commands.Remote -> Html Msg\nviewRemoteListContainer model remotes =\n Grid.row []\n [ Grid.col [ Col.lg1, Col.attrs [ class \"d-none d-lg-block\" ] ] []\n , Grid.col [ Col.xs12, Col.lg10 ]\n [ Util.viewAlert AlertMsg model.alert\n , viewRemoteList model remotes\n , div [ class \"text-left\" ]\n [ Button.button\n [ Button.roleLink\n , Button.attrs\n [ onClick <| RemoteAddMsg RemoteAdd.show\n , disabled (not (List.member \"remotes.edit\" model.rights))\n ]\n ]\n [ span [ class \"fas fa-lg fa-plus\" ] []\n , text \" Add new\"\n ]\n ]\n ]\n , Grid.col [ Col.lg1, Col.attrs [ class \"d-none d-lg-block\" ] ] []\n ]\n\n\nview : Model -> Html Msg\nview model =\n case model.state of\n Loading ->\n text \"Still loading\"\n\n Failure err ->\n text (\"Failed to load remote list: \" ++ err)\n\n Success remotes ->\n Grid.row []\n [ Grid.col\n [ Col.lg12 ]\n [ Grid.row [ Row.attrs [ id \"main-header-row\" ] ]\n []\n , Grid.row [ Row.attrs [ id \"main-content-row\" ] ]\n [ Grid.col\n [ Col.xl10 ]\n [ h4 [ class \"text-center text-muted\" ] [ text \"Own data\" ]\n , br [] []\n , viewSelf model\n , br [] []\n , br [] []\n , br [] []\n , br [] []\n , h4 [ class \"text-center text-muted\" ] [ text \"Other remotes\" ]\n , br [] []\n , viewRemoteListContainer model remotes\n ]\n ]\n ]\n ]\n\n\nbuildModals : Model -> Html Msg\nbuildModals model =\n span []\n [ Html.map RemoteAddMsg (RemoteAdd.view model.remoteAddState)\n , Html.map RemoteRemoveMsg (RemoteRemove.view model.remoteRemoveState)\n , Html.map RemoteFolderMsg (RemoteFolders.view model.remoteFoldersState)\n ]\n\n\n\n-- SUBSCRIPTIONS:\n\n\nsubscriptions : Model -> Sub Msg\nsubscriptions model =\n Sub.batch\n [ Alert.subscriptions model.alert.vis AlertMsg\n , Sub.map RemoteAddMsg <| RemoteAdd.subscriptions model.remoteAddState\n , Sub.map RemoteRemoveMsg <| RemoteRemove.subscriptions model.remoteRemoveState\n , Sub.map RemoteFolderMsg <| RemoteFolders.subscriptions model.remoteFoldersState\n , Sub.batch\n (List.map\n (\\( name, state ) -> Dropdown.subscriptions state (ActionDropdownMsg name))\n (Dict.toList model.actionDropdowns)\n )\n , Sub.batch\n (List.map\n (\\( name, state ) -> Dropdown.subscriptions state (ConflictDropdownMsg name))\n (Dict.toList model.conflictDropdowns)\n )\n ]\n" }, { "path": "gateway/elm/src/Scroll.elm", "content": "port module Scroll exposing (ScreenData, hasHitBottom, scrollOrResize)\n\n\ntype alias ScreenData =\n { scrollTop : Int\n , pageHeight : Int\n , viewportHeight : Int\n , viewportWidth : Int\n }\n\n\nport scrollOrResize : (ScreenData -> msg) -> Sub msg\n\n\npercFloat : ScreenData -> Float\npercFloat data =\n toFloat (data.scrollTop * 100) / toFloat (data.pageHeight - data.viewportHeight)\n\n\nhasHitBottom : ScreenData -> Bool\nhasHitBottom data =\n percFloat data >= 95\n" }, { "path": "gateway/elm/src/Util.elm", "content": "module Util exposing\n ( AlertState\n , AlertType(..)\n , basename\n , buildAlert\n , defaultAlertState\n , dirname\n , formatLastModified\n , formatLastModifiedOwner\n , httpErrorToString\n , joinPath\n , monthToInt\n , prefixSlash\n , splitPath\n , urlEncodePath\n , urlPrefixToString\n , urlToPath\n , viewAlert\n , viewToggleSwitch\n )\n\nimport Bootstrap.Alert as Alert\nimport Bootstrap.Button as Button\nimport Bootstrap.Grid as Grid\nimport Bootstrap.Grid.Col as Col\nimport Bootstrap.Text as Text\nimport Html exposing (..)\nimport Html.Attributes exposing (..)\nimport Html.Events exposing (..)\nimport Http\nimport Time\nimport Url\n\n\nmonthToInt : Time.Month -> Int\nmonthToInt month =\n -- This feels stupid.\n case month of\n Time.Jan ->\n 1\n\n Time.Feb ->\n 2\n\n Time.Mar ->\n 3\n\n Time.Apr ->\n 4\n\n Time.May ->\n 5\n\n Time.Jun ->\n 6\n\n Time.Jul ->\n 7\n\n Time.Aug ->\n 8\n\n Time.Sep ->\n 9\n\n Time.Oct ->\n 10\n\n Time.Nov ->\n 11\n\n Time.Dec ->\n 12\n\n\nformatLastModifiedOwner : Time.Zone -> Time.Posix -> String -> Html.Html msg\nformatLastModifiedOwner z t owner =\n p [] [ text (formatLastModified z t), span [ class \"text-muted\" ] [ text \" by \" ], text owner ]\n\n\nformatLastModified : Time.Zone -> Time.Posix -> String\nformatLastModified z t =\n String.join \" \"\n -- Day portion:\n [ String.join\n \"/\"\n [ Time.toDay z t |> String.fromInt\n , Time.toMonth z t |> monthToInt |> String.fromInt\n , Time.toYear z t |> String.fromInt\n ]\n\n -- Time portion:\n , String.join \":\"\n [ Time.toHour z t |> String.fromInt |> String.padLeft 2 '0'\n , Time.toMinute z t |> String.fromInt |> String.padLeft 2 '0'\n , Time.toSecond z t |> String.fromInt |> String.padLeft 2 '0'\n ]\n ]\n\n\nsplitPath : String -> List String\nsplitPath path =\n List.filter (\\s -> String.length s > 0) (String.split \"/\" path)\n\n\njoinPath : List String -> String\njoinPath paths =\n \"/\" ++ String.join \"/\" (List.foldr (++) [] (List.map splitPath paths))\n\n\nurlToPath : Url.Url -> String\nurlToPath url =\n let\n decodeUrlPart =\n \\e ->\n case Url.percentDecode e of\n Just val ->\n val\n\n Nothing ->\n \"\"\n in\n case splitPath url.path of\n [] ->\n \"/\"\n\n _ :: xs ->\n \"/\" ++ String.join \"/\" (List.map decodeUrlPart xs)\n\n\nbasename : String -> String\nbasename path =\n let\n split =\n List.reverse (splitPath path)\n in\n case split of\n [] ->\n \"/\"\n\n x :: _ ->\n x\n\n\nprefixSlash : String -> String\nprefixSlash path =\n if String.startsWith \"/\" path then\n path\n\n else\n \"/\" ++ path\n\n\ndirname : String -> String\ndirname path =\n let\n split =\n splitPath path\n in\n case split of\n [] ->\n \"/\"\n\n _ ->\n joinPath <| List.take (List.length split - 1) split\n\n\nbuildAlert : Alert.Visibility -> (Alert.Visibility -> msg) -> (Alert.Config msg -> Alert.Config msg) -> String -> String -> Html msg\nbuildAlert visibility msg severity title message =\n Alert.config\n |> Alert.dismissableWithAnimation msg\n |> severity\n |> Alert.children\n [ if String.length title > 0 then\n Alert.h4 [] [ text title ]\n\n else\n text \"\"\n , text message\n ]\n |> Alert.view visibility\n\n\nhttpErrorToString : Http.Error -> String\nhttpErrorToString err =\n case err of\n Http.BadUrl msg ->\n \"Bad url: \" ++ msg\n\n Http.Timeout ->\n \"Timeout\"\n\n Http.NetworkError ->\n \"Network error\"\n\n Http.BadStatus status ->\n \"Bad status: \" ++ String.fromInt status\n\n Http.BadBody msg ->\n \"Could not decode body: \" ++ msg\n\n\nurlPrefixToString : Url.Url -> String\nurlPrefixToString url =\n (case url.protocol of\n Url.Https ->\n \"https://\"\n\n Url.Http ->\n \"http://\"\n )\n ++ url.host\n ++ (case url.port_ of\n Just port_ ->\n \":\" ++ String.fromInt port_\n\n Nothing ->\n \"\"\n )\n ++ \"/\"\n\n\nurlEncodePath : String -> String\nurlEncodePath path =\n joinPath (List.map Url.percentEncode (splitPath path))\n\n\nviewToggleSwitch : (Bool -> msg) -> String -> Bool -> Bool -> Html msg\nviewToggleSwitch toMsg message isChecked isDisabled =\n span []\n [ span []\n [ label\n [ class \"toggle-switch\"\n , disabled isDisabled\n , if isDisabled then\n class \"toggle-switch-disabled\"\n\n else\n class \"\"\n ]\n [ input [ type_ \"checkbox\", onCheck toMsg, checked isChecked, disabled isDisabled ] []\n , span [ class \"toggle-slider toggle-round\" ] []\n ]\n ]\n , span\n [ class \"text-muted\" ]\n [ text (\" \" ++ message) ]\n ]\n\n\n\n-- ALERT UTILS\n\n\ntype AlertType\n = Danger\n | Success\n | Info\n\n\ntype alias AlertState =\n { message : String\n , typ : AlertType\n , vis : Alert.Visibility\n }\n\n\ndefaultAlertState : AlertState\ndefaultAlertState =\n { message = \"\"\n , typ = Info\n , vis = Alert.closed\n }\n\n\niconFromAlertType : AlertType -> Html msg\niconFromAlertType typ =\n case typ of\n Danger ->\n span [ class \"fas fa-xs fa-exclamation-circle\" ] []\n\n Success ->\n span [ class \"fas fa-xs fa-check\" ] []\n\n _ ->\n text \"\"\n\n\nvisualFromAlertType : AlertType -> (Alert.Config msg -> Alert.Config msg)\nvisualFromAlertType typ =\n case typ of\n Danger ->\n Alert.danger\n\n Success ->\n Alert.success\n\n _ ->\n Alert.info\n\n\nviewAlert : (Alert.Visibility -> msg) -> AlertState -> Html msg\nviewAlert toMsg alert =\n Alert.config\n |> Alert.dismissableWithAnimation toMsg\n |> visualFromAlertType alert.typ\n |> Alert.children\n [ Grid.row []\n [ Grid.col [ Col.xs10 ]\n [ iconFromAlertType alert.typ\n , text (\" \" ++ alert.message)\n ]\n , Grid.col [ Col.xs2, Col.textAlign Text.alignXsRight ]\n [ Button.button\n [ Button.roleLink\n , Button.attrs\n [ class \"notification-close-btn\"\n , onClick (toMsg Alert.closed)\n ]\n ]\n [ span [ class \"fas fa-xs fa-times\" ] [] ]\n ]\n ]\n ]\n |> Alert.view alert.vis\n" }, { "path": "gateway/elm/src/Websocket.elm", "content": "port module Websocket exposing (incoming, open)\n\n\nport incoming : (String -> msg) -> Sub msg\n\n\nport open : () -> Cmd msg\n" }, { "path": "gateway/endpoints/all_dirs.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n\t\"sort\"\n\t\"strings\"\n\n\t\"github.com/sahib/brig/gateway/db\"\n)\n\n// AllDirsHandler implements http.Handler.\n// This endpoint returns all directories that the client may see.\n// It is used in the client to offer the user a list of directories\n// to move or copy files to.\ntype AllDirsHandler struct {\n\t*State\n}\n\n// NewAllDirsHandler returns a new AllDirsHandler.\nfunc NewAllDirsHandler(s *State) *AllDirsHandler {\n\treturn &AllDirsHandler{State: s}\n}\n\n// AllDirsResponse is the response sent to the client.\ntype AllDirsResponse struct {\n\tSuccess bool `json:\"success\"`\n\tPaths []string `json:\"paths\"`\n}\n\nfunc (ah *AllDirsHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tif user := getUserName(ah.store, w, r); user == \"\" {\n\t\tjsonifyErrf(w, http.StatusForbidden, \"bad user\")\n\t\treturn\n\t}\n\n\tif !checkRights(w, r, db.RightFsView) {\n\t\treturn\n\t}\n\n\tnodes, err := ah.fs.List(\"/\", -1)\n\tif err != nil {\n\t\tjsonifyErrf(w, http.StatusInternalServerError, \"failed to list\")\n\t\treturn\n\t}\n\n\tpaths := []string{}\n\tfor _, node := range nodes {\n\t\tif !node.IsDir || !ah.validatePath(node.Path, w, r) {\n\t\t\tcontinue\n\t\t}\n\n\t\tpaths = append(paths, node.Path)\n\t}\n\n\t// Sort dirs before files and sort each part alphabetically\n\tsort.Slice(paths, func(i, j int) bool {\n\t\treturn strings.ToLower(paths[i]) < strings.ToLower(paths[j])\n\t})\n\n\tjsonify(w, http.StatusOK, &AllDirsResponse{\n\t\tSuccess: true,\n\t\tPaths: paths,\n\t})\n}\n" }, { "path": "gateway/endpoints/all_dirs_test.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestAllDirsSuccess(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\ts.mustChangeFolders(t, \"/a\")\n\n\t\trequire.Nil(t, s.fs.Mkdir(\"/a/b/c\", true))\n\t\trequire.Nil(t, s.fs.Mkdir(\"/d/e/f\", true))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewAllDirsHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/all_dirs\",\n\t\t\tnil,\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tallDirsResp := &AllDirsResponse{}\n\t\tmustDecodeBody(t, resp.Body, allDirsResp)\n\t\trequire.Equal(t, true, allDirsResp.Success)\n\t\trequire.Equal(t, []string{\"/a\", \"/a/b\", \"/a/b/c\"}, allDirsResp.Paths)\n\t})\n}\n" }, { "path": "gateway/endpoints/copy.go", "content": "package endpoints\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\n\t\"github.com/sahib/brig/gateway/db\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// CopyHandler implements http.Handler.\ntype CopyHandler struct {\n\t*State\n}\n\n// NewCopyHandler creates a new copy handler.\nfunc NewCopyHandler(s *State) *CopyHandler {\n\treturn &CopyHandler{State: s}\n}\n\n// CopyRequest is the request that can be send to this endpoint.\ntype CopyRequest struct {\n\t// Source is the path to the old node.\n\tSource string `json:\"source\"`\n\t// Destination is the path of the new node.\n\tDestination string `json:\"destination\"`\n}\n\nfunc (ch *CopyHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tif !checkRights(w, r, db.RightFsEdit) {\n\t\treturn\n\t}\n\n\tcopyReq := CopyRequest{}\n\tif err := json.NewDecoder(r.Body).Decode(©Req); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"bad json\")\n\t\treturn\n\t}\n\n\tsrc := prefixRoot(copyReq.Source)\n\tdst := prefixRoot(copyReq.Destination)\n\n\tif !ch.validatePath(src, w, r) {\n\t\tjsonifyErrf(w, http.StatusUnauthorized, \"source path forbidden\")\n\t\treturn\n\t}\n\n\tif !ch.validatePath(dst, w, r) {\n\t\tjsonifyErrf(w, http.StatusUnauthorized, \"destination path forbidden\")\n\t\treturn\n\t}\n\n\tif err := ch.fs.Copy(src, dst); err != nil {\n\t\tlog.Debugf(\"failed to copy %s -> %s: %v\", src, dst, err)\n\t\tjsonifyErrf(w, http.StatusInternalServerError, \"failed to copy\")\n\t\treturn\n\t}\n\n\tmsg := fmt.Sprintf(\"copied »%s« to »%s«\", src, dst)\n\tif !ch.commitChange(msg, w, r) {\n\t\treturn\n\t}\n\n\tjsonifySuccess(w)\n}\n" }, { "path": "gateway/endpoints/copy_test.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\ntype copyResponse struct {\n\tSuccess bool `json:\"success\"`\n}\n\nfunc TestCopySuccess(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\trequire.Nil(t, s.fs.Mkdir(\"/hinz\", true))\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewCopyHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/copy\",\n\t\t\t&CopyRequest{\n\t\t\t\tSource: \"/hinz\",\n\t\t\t\tDestination: \"/kunz\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tcopyResp := ©Response{}\n\t\tmustDecodeBody(t, resp.Body, copyResp)\n\t\trequire.Equal(t, true, copyResp.Success)\n\n\t\thinzInfo, err := s.fs.Stat(\"/hinz\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, \"/hinz\", hinzInfo.Path)\n\n\t\tkunzInfo, err := s.fs.Stat(\"/kunz\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, \"/kunz\", kunzInfo.Path)\n\t})\n}\n\nfunc TestCopyDisallowedSource(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\ts.mustChangeFolders(t, \"/kunz\")\n\t\trequire.Nil(t, s.fs.Mkdir(\"/hinz\", true))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewCopyHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/copy\",\n\t\t\t&CopyRequest{\n\t\t\t\tSource: \"/hinz\",\n\t\t\t\tDestination: \"/kunz\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusUnauthorized, resp.StatusCode)\n\t})\n}\n\nfunc TestCopyDisallowedDest(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\ts.mustChangeFolders(t, \"/hinz\")\n\t\trequire.Nil(t, s.fs.Mkdir(\"/hinz\", true))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewCopyHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/copy\",\n\t\t\t&CopyRequest{\n\t\t\t\tSource: \"/hinz\",\n\t\t\t\tDestination: \"/kunz\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusUnauthorized, resp.StatusCode)\n\t})\n}\n" }, { "path": "gateway/endpoints/deleted.go", "content": "package endpoints\n\nimport (\n\t\"encoding/json\"\n\t\"net/http\"\n\t\"sort\"\n\t\"strings\"\n\n\t\"github.com/sahib/brig/catfs\"\n\t\"github.com/sahib/brig/gateway/db\"\n\t\"github.com/sahib/brig/util\"\n)\n\n// DeletedPathsHandler implements http.Handler.\n// This endpoint returns all directories that the client may see.\n// It is used in the client to offer the user a list of directories\n// to move or copy files to.\ntype DeletedPathsHandler struct {\n\t*State\n}\n\n// NewDeletedPathsHandler returns a new DeletedPathsHandler.\nfunc NewDeletedPathsHandler(s *State) *DeletedPathsHandler {\n\treturn &DeletedPathsHandler{State: s}\n}\n\n// DeletedPathsResponse is the response sent to the client.\ntype DeletedPathsResponse struct {\n\tSuccess bool `json:\"success\"`\n\tEntries []*StatInfo `json:\"entries\"`\n}\n\n// DeletedRequest is the data sent to this endpoint.\ntype DeletedRequest struct {\n\tOffset int64 `json:\"offset\"`\n\tLimit int64 `json:\"limit\"`\n\tFilter string `json:\"filter\"`\n}\n\nfunc matchEntry(info *catfs.StatInfo, filter string) bool {\n\tif filter == \"\" {\n\t\treturn true\n\t}\n\n\treturn strings.Contains(strings.ToLower(info.Path), filter)\n}\n\nfunc (dh *DeletedPathsHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tif !checkRights(w, r, db.RightFsView) {\n\t\treturn\n\t}\n\n\tdelReq := DeletedRequest{}\n\tif err := json.NewDecoder(r.Body).Decode(&delReq); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"bad json\")\n\t\treturn\n\t}\n\n\tif delReq.Offset < 0 {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"negative offset\")\n\t\treturn\n\t}\n\n\tnodes, err := dh.fs.DeletedNodes(\"/\")\n\tif err != nil {\n\t\tjsonifyErrf(w, http.StatusInternalServerError, \"failed to list\")\n\t\treturn\n\t}\n\n\tfilter := strings.ToLower(delReq.Filter)\n\tfilteredNodes := []*catfs.StatInfo{}\n\tfor _, node := range nodes {\n\t\tif !matchEntry(node, filter) {\n\t\t\tcontinue\n\t\t}\n\n\t\tfilteredNodes = append(filteredNodes, node)\n\t}\n\n\tentries := []*StatInfo{}\n\tif delReq.Offset >= int64(len(filteredNodes)) {\n\t\tjsonify(w, http.StatusOK, &DeletedPathsResponse{\n\t\t\tSuccess: true,\n\t\t\tEntries: entries,\n\t\t})\n\t\treturn\n\t}\n\n\tfilteredNodes = filteredNodes[delReq.Offset:]\n\tif delReq.Limit >= 0 {\n\t\tfilteredNodes = filteredNodes[:util.Min64(int64(len(filteredNodes)), delReq.Limit)]\n\t}\n\n\tfor _, node := range filteredNodes {\n\t\tif !dh.validatePath(node.Path, w, r) {\n\t\t\tcontinue\n\t\t}\n\n\t\tentries = append(entries, toExternalStatInfo(node))\n\t}\n\n\t// Sort dirs before files and sort each part alphabetically\n\tsort.Slice(entries, func(i, j int) bool {\n\t\tif entries[i].IsDir != entries[j].IsDir {\n\t\t\treturn entries[i].IsDir\n\t\t}\n\n\t\treturn strings.ToLower(entries[i].Path) < strings.ToLower(entries[j].Path)\n\t})\n\n\tjsonify(w, http.StatusOK, &DeletedPathsResponse{\n\t\tSuccess: true,\n\t\tEntries: entries,\n\t})\n}\n" }, { "path": "gateway/endpoints/deleted_test.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestDeletedPathSuccess(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\ts.mustChangeFolders(t, \"/a\")\n\t\trequire.Nil(t, s.fs.Touch(\"/a/b/c1\"))\n\t\trequire.Nil(t, s.fs.Touch(\"/a/b/c2\"))\n\t\trequire.Nil(t, s.fs.Touch(\"/d/e/f1\"))\n\t\trequire.Nil(t, s.fs.Touch(\"/d/e/f2\"))\n\n\t\trequire.Nil(t, s.fs.MakeCommit(\"add\"))\n\n\t\trequire.Nil(t, s.fs.Remove(\"/a/b/c1\"))\n\t\trequire.Nil(t, s.fs.Remove(\"/a/b/c2\"))\n\t\trequire.Nil(t, s.fs.Remove(\"/d/e/f1\"))\n\t\trequire.Nil(t, s.fs.Remove(\"/d/e/f2\"))\n\t\trequire.Nil(t, s.fs.MakeCommit(\"rm\"))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewDeletedPathsHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/deleted\",\n\t\t\t&DeletedRequest{\n\t\t\t\tOffset: 0,\n\t\t\t\tLimit: -1,\n\t\t\t\tFilter: \"\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tdeletedResp := &DeletedPathsResponse{}\n\t\tmustDecodeBody(t, resp.Body, deletedResp)\n\t\trequire.Equal(t, true, deletedResp.Success)\n\t\trequire.Equal(t, 2, len(deletedResp.Entries))\n\n\t\tpaths := []string{}\n\t\tfor _, entry := range deletedResp.Entries {\n\t\t\tpaths = append(paths, entry.Path)\n\t\t}\n\n\t\trequire.Equal(\n\t\t\tt,\n\t\t\t[]string{\"/a/b/c1\", \"/a/b/c2\"},\n\t\t\tpaths,\n\t\t)\n\t})\n}\n" }, { "path": "gateway/endpoints/events.go", "content": "package endpoints\n\nimport (\n\t\"context\"\n\t\"net/http\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/gorilla/websocket\"\n\t\"github.com/sahib/brig/events\"\n\t\"github.com/sahib/brig/gateway/db\"\n\t\"github.com/sahib/brig/gateway/remotesapi\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\nvar upgrader = websocket.Upgrader{\n\tReadBufferSize: 1024,\n\tWriteBufferSize: 1024,\n}\n\n// EventsHandler implements http.Handler\ntype EventsHandler struct {\n\tmu sync.Mutex\n\tid int\n\tchs map[int]chan string\n\trapi remotesapi.RemotesAPI\n\tevListener *events.Listener\n\tchangeOnce sync.Once\n\n\t// only true while unit tests.\n\t// circumvents the right check,\n\t// that can't be mocked away easily.\n\ttesting bool\n}\n\n// NewEventsHandler returns a new EventsHandler\nfunc NewEventsHandler(rapi remotesapi.RemotesAPI, ev *events.Listener) *EventsHandler {\n\thdl := &EventsHandler{\n\t\tchs: make(map[int]chan string),\n\t\trapi: rapi,\n\t}\n\n\tif ev != nil {\n\t\t// Incoming events from our own node:\n\t\tev.RegisterEventHandler(events.FsEvent, true, func(ev *events.Event) {\n\t\t\tctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)\n\t\t\tdefer cancel()\n\n\t\t\thdl.notify(ctx, \"fs\", true, false)\n\t\t})\n\n\t\t// Incoming events from other nodes:\n\t\tev.RegisterEventHandler(events.FsEvent, false, func(ev *events.Event) {\n\t\t\tctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)\n\t\t\tdefer cancel()\n\n\t\t\thdl.notify(ctx, \"fs\", false, false)\n\t\t})\n\n\t\thdl.evListener = ev\n\t}\n\treturn hdl\n}\n\n// Notify sends `msg` to all connected clients, but stops in case `ctx`\n// was canceled before sending it all.\nfunc (eh *EventsHandler) Notify(ctx context.Context, msg string) error {\n\treturn eh.notify(ctx, msg, true, true)\n}\n\nfunc (eh *EventsHandler) notify(ctx context.Context, msg string, isOwnEvent, triggerPublish bool) error {\n\teh.mu.Lock()\n\tchs := []chan string{}\n\tfor _, ch := range eh.chs {\n\t\tchs = append(chs, ch)\n\t}\n\teh.mu.Unlock()\n\n\tfor _, ch := range chs {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\treturn ctx.Err()\n\t\tcase ch <- msg:\n\t\t\tcontinue\n\t\t}\n\t}\n\n\t// We can only trigger fs events in the gateway:\n\tevent := events.Event{\n\t\tType: events.FsEvent,\n\t}\n\n\tif !isOwnEvent && triggerPublish && eh.evListener != nil {\n\t\treturn eh.evListener.PublishEvent(event)\n\t}\n\n\treturn nil\n}\n\n// Shutdown closes all open websockets.\nfunc (eh *EventsHandler) Shutdown() {\n\teh.mu.Lock()\n\tdefer eh.mu.Unlock()\n\n\tfor _, ch := range eh.chs {\n\t\tclose(ch)\n\t}\n}\n\nfunc (eh *EventsHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tif !eh.testing {\n\t\tif !checkRights(w, r, db.RightFsView) {\n\t\t\treturn\n\t\t}\n\t}\n\n\tconn, err := upgrader.Upgrade(w, r, nil)\n\tif err != nil {\n\t\tlog.Warningf(\"failed to upgrade to websocket: %v\", err)\n\t\treturn\n\t}\n\n\t// We setup the on change handler only here,\n\t// since calling OnChange in init might deadlock\n\t// since the real implementation might call Repo()\n\teh.changeOnce.Do(func() {\n\t\teh.rapi.OnChange(func() {\n\t\t\tctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)\n\t\t\tdefer cancel()\n\n\t\t\teh.Notify(ctx, \"remotes\")\n\t\t})\n\t})\n\n\teh.mu.Lock()\n\tid := eh.id\n\teh.id++\n\tch := make(chan string, 20)\n\teh.chs[id] = ch\n\teh.mu.Unlock()\n\n\tdefer func() {\n\t\teh.mu.Lock()\n\t\tdelete(eh.chs, id)\n\t\teh.mu.Unlock()\n\t}()\n\n\tdefer conn.Close()\n\n\tfor {\n\t\tselect {\n\t\tcase msg, ok := <-ch:\n\t\t\tif !ok {\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tif err := conn.WriteMessage(websocket.TextMessage, []byte(msg)); err != nil {\n\t\t\t\tlog.Debugf(\"failed to write to websocket, closing: %v\", err)\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}\n}\n" }, { "path": "gateway/endpoints/events_test.go", "content": "package endpoints\n\nimport (\n\t\"context\"\n\t\"net/http\"\n\t\"runtime\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/gorilla/websocket\"\n\t\"github.com/posener/wstest\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestEvents(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\t// This is stupid. I couldn't get DialContext()\n\t\t// to pass the user value to the actual handler.\n\t\t// Pretty sure it was a problem on my side though...\n\t\ts.evHdl.testing = true\n\n\t\t// This call evHdl.ServeHTTP when sending something on conn.\n\t\tdialer := wstest.NewDialer(s.evHdl)\n\n\t\tctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)\n\t\tdefer cancel()\n\n\t\tconn, resp, err := dialer.DialContext(ctx, \"ws://whatever/ws\", nil)\n\t\trequire.Nil(t, err)\n\n\t\tif got, want := resp.StatusCode, http.StatusSwitchingProtocols; got != want {\n\t\t\tt.Fatalf(\"resp.StatusCode = %q, want %q\", got, want)\n\t\t}\n\n\t\tgo func() {\n\t\t\t// give it a little time so ServeHTTP() of the events handler\n\t\t\t// can reach the \"please notify me now\" stage.\n\t\t\ttime.Sleep(100 * time.Millisecond)\n\n\t\t\t// trigger an event:\n\t\t\tresp := s.mustRun(\n\t\t\t\tt,\n\t\t\t\tNewMkdirHandler(s.State),\n\t\t\t\t\"POST\",\n\t\t\t\t\"http://localhost:5000/api/v0/events\",\n\t\t\t\t&MkdirRequest{\n\t\t\t\t\tPath: \"/test\",\n\t\t\t\t},\n\t\t\t)\n\t\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\t\t}()\n\n\t\tdone := make(chan bool)\n\n\t\tgo func() {\n\t\t\ttyp, data, err := conn.ReadMessage()\n\t\t\trequire.Nil(t, err)\n\t\t\trequire.Equal(t, websocket.TextMessage, typ)\n\t\t\trequire.Equal(t, []byte(\"fs\"), data)\n\t\t\tdone <- true\n\t\t}()\n\n\t\tselect {\n\t\tcase <-done:\n\t\tcase <-time.After(10 * time.Second):\n\t\t\tbuf := make([]byte, 1<<20)\n\t\t\tstacklen := runtime.Stack(buf, true)\n\t\t\tt.Logf(string(buf[:stacklen]))\n\t\t\tt.Fatalf(\"test took too long\")\n\t\t}\n\t})\n}\n" }, { "path": "gateway/endpoints/get.go", "content": "package endpoints\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"net/url\"\n\t\"path\"\n\t\"strconv\"\n\t\"strings\"\n\n\t\"github.com/sahib/brig/catfs\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\t\"github.com/sahib/brig/catfs/mio\"\n\t\"github.com/sahib/brig/gateway/db\"\n\t\"github.com/sahib/brig/util\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// GetHandler implements http.Handler\ntype GetHandler struct {\n\t*State\n}\n\n// NewGetHandler returns a new GetHandler\nfunc NewGetHandler(s *State) *GetHandler {\n\treturn &GetHandler{State: s}\n}\n\nfunc mimeTypeFromStream(stream mio.Stream) (io.ReadSeeker, string) {\n\thdr, newStream, err := util.PeekHeader(stream, 512)\n\tif err != nil {\n\t\treturn stream, \"application/octet-stream\"\n\t}\n\n\treturn newStream, http.DetectContentType(hdr)\n}\n\n// setContentDisposition sets the Content-Disposition header, based on\n// the content we are serving. It tells a browser if it should open\n// a save dialog or display it inline (and how)\nfunc setContentDisposition(info *catfs.StatInfo, hdr http.Header, dispoType string) {\n\tbasename := path.Base(info.Path)\n\tif info.IsDir {\n\t\tif basename == \"/\" {\n\t\t\tbasename = \"root\"\n\t\t}\n\n\t\tbasename += \".tar\"\n\t}\n\n\thdr.Set(\n\t\t\"Content-Disposition\",\n\t\tfmt.Sprintf(\n\t\t\t\"%s; filename*=UTF-8''%s\",\n\t\t\tdispoType,\n\t\t\turl.QueryEscape(basename),\n\t\t),\n\t)\n}\n\nfunc (gh *GetHandler) checkBasicAuth(nodePath string, w http.ResponseWriter, r *http.Request) bool {\n\tname, pass, ok := r.BasicAuth()\n\n\t// No basic auth sent. If a browser send the request: ask him to\n\t// show a user/password form that gives a chance to change that.\n\tif !ok {\n\t\tw.Header().Set(\"WWW-Authenticate\", \"Basic realm=\\\"brig gateway\\\"\")\n\t\treturn false\n\t}\n\n\t// Check is the basic auth credentials are valid.\n\tuser, err := gh.userDb.Get(name)\n\tif err != nil {\n\t\treturn false\n\t}\n\n\thasRight := false\n\tfor _, right := range user.Rights {\n\t\tif right == db.RightDownload {\n\t\t\thasRight = true\n\t\t\tbreak\n\t\t}\n\t}\n\n\tif !hasRight {\n\t\treturn false\n\t}\n\n\tisValid, err := user.CheckPassword(pass)\n\tif !isValid {\n\t\tif err != nil {\n\t\t\tlog.Warningf(\"get: failed to check password: %v\", err)\n\t\t}\n\n\t\treturn false\n\t}\n\n\t// Check again if this user has access to the path:\n\tif !gh.validatePathForUser(nodePath, user, w, r) {\n\t\treturn false\n\t}\n\n\treturn true\n}\n\nfunc (gh *GetHandler) checkDownloadRight(w http.ResponseWriter, r *http.Request) bool {\n\tname := getUserName(gh.store, w, r)\n\tif name == \"\" {\n\t\treturn false\n\t}\n\n\treturn gh.checkDownloadRightByName(name, w, r)\n}\n\nfunc (gh *GetHandler) checkDownloadRightByName(name string, w http.ResponseWriter, r *http.Request) bool {\n\tuser, err := gh.userDb.Get(name)\n\tif err != nil {\n\t\treturn false\n\t}\n\n\tfor _, right := range user.Rights {\n\t\tif right == db.RightDownload {\n\t\t\treturn true\n\t\t}\n\t}\n\n\treturn false\n}\n\nfunc (gh *GetHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\t// get the file nodePath including the leading slash:\n\tfullURL := r.URL.EscapedPath()\n\tnodePath, err := url.PathUnescape(fullURL[4:])\n\tif nodePath == \"\" {\n\t\tnodePath = \"/\"\n\t}\n\n\tif err != nil {\n\t\tlog.Debugf(\"received malformed url: %s\", fullURL)\n\t\thttp.Error(w, \"malformed url\", http.StatusBadRequest)\n\t\treturn\n\t}\n\n\tif gh.cfg.Bool(\"auth.anon_allowed\") {\n\t\t// validatePath will check if the user is actually logged in\n\t\t// and may access the path in question. The login could come\n\t\t// from a previous login to the UI (the /get endpoint could be used separately)\n\t\tif !gh.validatePath(nodePath, w, r) {\n\t\t\t// If the user was not previously logged into the UI,\n\t\t\t// we also accept basic auth for this endpoint.\n\t\t\t// This way hyperlinks can be shared without having to login.\n\t\t\t// Using HTTPS here is strongly recommended.\n\t\t\tif !gh.checkBasicAuth(nodePath, w, r) {\n\t\t\t\thttp.Error(w, \"not authorized\", http.StatusUnauthorized)\n\t\t\t\treturn\n\t\t\t}\n\t\t} else {\n\t\t\t// Check if the user allowed the anon user to download files.\n\t\t\tanonName := gh.cfg.String(\"auth.anon_user\")\n\t\t\tif !gh.checkDownloadRightByName(anonName, w, r) {\n\t\t\t\thttp.Error(w, \"insufficient rights\", http.StatusUnauthorized)\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\n\t\t// All good. Proceed with the content.\n\t} else {\n\t\tif !gh.validatePath(nodePath, w, r) {\n\t\t\thttp.Error(w, \"insufficient rights\", http.StatusUnauthorized)\n\t\t\treturn\n\t\t}\n\t\tif !gh.checkDownloadRight(w, r) {\n\t\t\thttp.Error(w, \"insufficient rights for anon\", http.StatusUnauthorized)\n\t\t\treturn\n\t\t}\n\t}\n\n\tinfo, err := gh.fs.Stat(nodePath)\n\tif err != nil {\n\t\t// Handle a bad nodePath more explicit:\n\t\tif ie.IsNoSuchFileError(err) {\n\t\t\thttp.Error(w, \"not found\", http.StatusNotFound)\n\t\t\treturn\n\t\t}\n\n\t\tlog.Errorf(\"gateway: failed to stat %s: %v\", nodePath, err)\n\t\thttp.Error(w, \"failed to stat file\", http.StatusInternalServerError)\n\t\treturn\n\t}\n\n\thdr := w.Header()\n\thdr.Set(\"ETag\", info.ContentHash.B58String())\n\thdr.Set(\"Last-Modified\", info.ModTime.Format(http.TimeFormat))\n\n\tif info.IsDir {\n\t\tparams := r.URL.Query()\n\t\tincludes := params[\"include\"]\n\n\t\tfilter := func(info *catfs.StatInfo) bool {\n\t\t\tif len(includes) == 0 {\n\t\t\t\treturn true\n\t\t\t}\n\n\t\t\tfor _, include := range includes {\n\t\t\t\tif strings.HasPrefix(info.Path, include) {\n\t\t\t\t\treturn true\n\t\t\t\t}\n\t\t\t}\n\n\t\t\treturn false\n\t\t}\n\n\t\tsetContentDisposition(info, hdr, \"attachment\")\n\t\tif err := gh.fs.Tar(nodePath, w, filter); err != nil {\n\t\t\tlog.Errorf(\"gateway: failed to stream %s: %v\", nodePath, err)\n\t\t\thttp.Error(w, \"failed to stream\", http.StatusInternalServerError)\n\t\t\treturn\n\t\t}\n\t} else {\n\t\tstream, err := gh.fs.Cat(nodePath)\n\t\tif err != nil {\n\t\t\tlog.Errorf(\"gateway: failed to stream %s: %v\", nodePath, err)\n\t\t\thttp.Error(w, \"failed to stream\", http.StatusInternalServerError)\n\t\t\treturn\n\t\t}\n\n\t\tprefixStream, mimeType := mimeTypeFromStream(stream)\n\t\thdr.Set(\"Content-Type\", mimeType)\n\t\thdr.Set(\"Content-Length\", strconv.FormatUint(info.Size, 10))\n\n\t\tisDirectDownload := r.URL.Query().Get(\"direct\") == \"yes\"\n\n\t\t// Set the content disposition to inline if it looks like something viewable.\n\t\tif mimeType == \"application/octet-stream\" || isDirectDownload {\n\t\t\tsetContentDisposition(info, hdr, \"attachment\")\n\t\t} else {\n\t\t\tsetContentDisposition(info, hdr, \"inline\")\n\t\t}\n\n\t\thttp.ServeContent(w, r, path.Base(info.Path), info.ModTime, prefixStream)\n\t}\n}\n" }, { "path": "gateway/endpoints/get_test.go", "content": "package endpoints\n\nimport (\n\t\"bytes\"\n\t\"io/ioutil\"\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestGetEndpointSuccess(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\tfileData := []byte(\"HelloWorld\")\n\t\trequire.Nil(t, s.fs.Stage(\"/file\", bytes.NewReader(fileData)))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewGetHandler(s.State),\n\t\t\t\"GET\",\n\t\t\t\"http://localhost:5000/get/file\",\n\t\t\tnil,\n\t\t)\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\t\tdata, err := ioutil.ReadAll(resp.Body)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, fileData, data)\n\t})\n}\n\nfunc TestGetEndpointDisallowed(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\tfileData := []byte(\"HelloWorld\")\n\t\trequire.Nil(t, s.fs.Stage(\"/file\", bytes.NewReader(fileData)))\n\t\ts.mustChangeFolders(t, \"/public\")\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewGetHandler(s.State),\n\t\t\t\"GET\",\n\t\t\t\"http://localhost:5000/get/file\",\n\t\t\tnil,\n\t\t)\n\n\t\trequire.Equal(t, http.StatusUnauthorized, resp.StatusCode)\n\t})\n}\n" }, { "path": "gateway/endpoints/history.go", "content": "package endpoints\n\nimport (\n\t\"encoding/json\"\n\t\"net/http\"\n\n\t\"github.com/sahib/brig/catfs\"\n\t\"github.com/sahib/brig/gateway/db\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// HistoryHandler implements http.Handler\ntype HistoryHandler struct {\n\t*State\n}\n\n// NewHistoryHandler returns a new HistoryHandler\nfunc NewHistoryHandler(s *State) *HistoryHandler {\n\treturn &HistoryHandler{State: s}\n}\n\n// HistoryRequest is the request sent to this endpoint.\ntype HistoryRequest struct {\n\tPath string `json:\"path\"`\n}\n\n// Commit is the same as catfs.Commit, but JSON friendly\n// and with some omitted fields that are not used by the client.\ntype Commit struct {\n\tDate int64 `json:\"date\"`\n\tMsg string `json:\"msg\"`\n\tTags []string `json:\"tags\"`\n\tHash string `json:\"hash\"`\n\tIndex int64 `json:\"index\"`\n}\n\n// HistoryEntry is one entry in the response.\ntype HistoryEntry struct {\n\tHead Commit `json:\"head\"`\n\tPath string `json:\"path\"`\n\tChange string `json:\"change\"`\n\tIsPinned bool `json:\"is_pinned\"`\n\tIsExplicit bool `json:\"is_explicit\"`\n}\n\n// HistoryResponse is the data that is sent back to the client.\ntype HistoryResponse struct {\n\tSuccess bool `json:\"success\"`\n\tEntries []HistoryEntry `json:\"entries\"`\n}\n\nfunc toExternalCommit(cmt *catfs.Commit) Commit {\n\text := Commit{}\n\text.Date = cmt.Date.Unix() * 1000\n\text.Hash = cmt.Hash.B58String()\n\text.Msg = cmt.Msg\n\text.Tags = cmt.Tags\n\text.Index = cmt.Index\n\n\t// Make sure we set an empty list,\n\t// otherwise .Tags gets serialized as null\n\t// which breaks frontend.\n\tif ext.Tags == nil {\n\t\text.Tags = []string{}\n\t}\n\treturn ext\n}\n\nfunc toExternalChange(c catfs.Change) HistoryEntry {\n\te := HistoryEntry{}\n\te.Change = c.Change\n\te.Head = toExternalCommit(c.Head)\n\te.Path = c.Path\n\te.IsPinned = c.IsPinned\n\te.IsExplicit = c.IsExplicit\n\treturn e\n}\n\nfunc (hh *HistoryHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tif !checkRights(w, r, db.RightFsView) {\n\t\treturn\n\t}\n\n\thistReq := HistoryRequest{}\n\tif err := json.NewDecoder(r.Body).Decode(&histReq); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"bad json\")\n\t\treturn\n\t}\n\n\tpath := prefixRoot(histReq.Path)\n\tif !hh.validatePath(path, w, r) {\n\t\tjsonifyErrf(w, http.StatusUnauthorized, \"path forbidden\")\n\t\treturn\n\t}\n\n\thist, err := hh.fs.History(path)\n\tif err != nil {\n\t\tlog.Debugf(\"failed to check history for %s: %v\", path, err)\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"failed to check history\")\n\t\treturn\n\t}\n\n\tentries := []HistoryEntry{}\n\tfor _, change := range hist {\n\t\t// Filter none changes, since they are only neat for debugging.\n\t\tif change.Change == \"none\" {\n\t\t\tcontinue\n\t\t}\n\n\t\tentries = append(entries, toExternalChange(change))\n\t}\n\n\tjsonify(w, http.StatusOK, &HistoryResponse{\n\t\tSuccess: true,\n\t\tEntries: entries,\n\t})\n}\n" }, { "path": "gateway/endpoints/history_test.go", "content": "package endpoints\n\nimport (\n\t\"bytes\"\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestHistoryEndpointSuccess(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\trequire.Nil(t, s.fs.Stage(\"/x\", bytes.NewReader([]byte(\"hello\"))))\n\t\trequire.Nil(t, s.fs.MakeCommit(\"hello\"))\n\t\trequire.Nil(t, s.fs.Stage(\"/x\", bytes.NewReader([]byte(\"world\"))))\n\t\trequire.Nil(t, s.fs.MakeCommit(\"world\"))\n\t\trequire.Nil(t, s.fs.Remove(\"/x\"))\n\t\trequire.Nil(t, s.fs.MakeCommit(\"remove\"))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewHistoryHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/history\",\n\t\t\t&HistoryRequest{\n\t\t\t\tPath: \"/x\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tdata := &HistoryResponse{}\n\t\tmustDecodeBody(t, resp.Body, &data)\n\t\trequire.Equal(t, true, data.Success)\n\n\t\tents := data.Entries\n\t\trequire.Len(t, ents, 3)\n\n\t\trequire.Equal(t, \"removed\", ents[0].Change)\n\t\trequire.Equal(t, \"/x\", ents[0].Path)\n\t\trequire.Equal(t, \"remove\", ents[0].Head.Msg)\n\t\trequire.Equal(t, []string{\"head\"}, ents[0].Head.Tags)\n\n\t\trequire.Equal(t, \"modified\", ents[1].Change)\n\t\trequire.Equal(t, \"/x\", ents[1].Path)\n\t\trequire.Equal(t, \"world\", ents[1].Head.Msg)\n\t\trequire.Equal(t, []string{}, ents[1].Head.Tags)\n\n\t\trequire.Equal(t, \"added\", ents[2].Change)\n\t\trequire.Equal(t, \"/x\", ents[2].Path)\n\t\trequire.Equal(t, \"hello\", ents[2].Head.Msg)\n\t\trequire.Equal(t, []string{\"init\"}, ents[2].Head.Tags)\n\t})\n}\n\nfunc TestHistoryEndpointForbidden(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\ts.mustChangeFolders(t, \"/public\")\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewHistoryHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/history\",\n\t\t\t&HistoryRequest{\n\t\t\t\tPath: \"/x\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusUnauthorized, resp.StatusCode)\n\t})\n}\n" }, { "path": "gateway/endpoints/index.go", "content": "package endpoints\n\nimport (\n\t\"html/template\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"net/http\"\n\t\"os\"\n\n\t\"github.com/gorilla/csrf\"\n\t\"github.com/phogolabs/parcello\"\n\tlog \"github.com/sirupsen/logrus\"\n\n\t// Include static resources:\n\t_ \"github.com/sahib/brig/gateway/templates\"\n)\n\n// IndexHandler implements http.Handler.\n// It serves index.html from either file or memory.\ntype IndexHandler struct {\n\t*State\n}\n\n// NewIndexHandler returns a new IndexHandler.\nfunc NewIndexHandler(s *State) *IndexHandler {\n\treturn &IndexHandler{State: s}\n}\n\nfunc (ih *IndexHandler) loadTemplateData() (io.ReadCloser, error) {\n\tif ih.cfg.Bool(\"ui.debug_mode\") {\n\t\treturn os.Open(\"./gateway/templates/index.html\")\n\t}\n\n\tmgr := parcello.ManagerAt(\"/\")\n\treturn mgr.Open(\"index.html\")\n}\n\nfunc (ih *IndexHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tfd, err := ih.loadTemplateData()\n\tif err != nil {\n\t\tjsonifyErrf(w, http.StatusInternalServerError, \"no index.html\")\n\t\treturn\n\t}\n\n\tdefer fd.Close()\n\n\tdata, err := ioutil.ReadAll(fd)\n\tif err != nil {\n\t\tjsonifyErrf(w, http.StatusInternalServerError, \"could not load template: %v\", err)\n\t\treturn\n\t}\n\n\tt, err := template.New(\"index\").Parse(string(data))\n\tif err != nil {\n\t\tlog.Errorf(\"could not parse template: %v\", err)\n\t\tjsonifyErrf(w, http.StatusInternalServerError, \"template contains errors\")\n\t\treturn\n\t}\n\n\twsScheme := \"ws://\"\n\tif r.TLS != nil {\n\t\twsScheme = \"wss://\"\n\t}\n\n\thttpScheme := \"http://\"\n\tif r.TLS != nil {\n\t\thttpScheme = \"https://\"\n\t}\n\n\terr = t.Execute(w, map[string]interface{}{\n\t\t\"csrfToken\": csrf.Token(r),\n\t\t\"wsAddr\": wsScheme + r.Host + \"/events\",\n\t\t\"httpAddr\": httpScheme + r.Host,\n\t})\n\n\tif err != nil {\n\t\tjsonifyErrf(w, http.StatusInternalServerError, \"could not execute template\")\n\t\treturn\n\t}\n}\n" }, { "path": "gateway/endpoints/log.go", "content": "package endpoints\n\nimport (\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"strings\"\n\n\t\"github.com/sahib/brig/catfs\"\n\t\"github.com/sahib/brig/gateway/db\"\n)\n\n// LogHandler implements http.Handler.\ntype LogHandler struct {\n\t*State\n}\n\n// NewLogHandler returns a new LogHandler\nfunc NewLogHandler(s *State) *LogHandler {\n\treturn &LogHandler{State: s}\n}\n\n// LogRequest is the data sent to this endpoint.\ntype LogRequest struct {\n\tOffset int64 `json:\"offset\"`\n\tLimit int64 `json:\"limit\"`\n\tFilter string `json:\"filter\"`\n}\n\n// LogResponse is the response sent back to the client.\ntype LogResponse struct {\n\tSuccess bool `json:\"success\"`\n\tHaveStagedChanges bool `json:\"have_staged_changes\"`\n\tCommits []Commit `json:\"commits\"`\n}\n\nfunc matchCommit(cmt *catfs.Commit, filter string) bool {\n\treturn strings.Contains(strings.ToLower(cmt.Msg), filter)\n}\n\nfunc (lh *LogHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tif !checkRights(w, r, db.RightFsView) {\n\t\treturn\n\t}\n\n\tlogReq := LogRequest{}\n\tif err := json.NewDecoder(r.Body).Decode(&logReq); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"bad json\")\n\t\treturn\n\t}\n\n\tstatus, err := lh.fs.CommitInfo(\"curr\")\n\tif err != nil {\n\t\tjsonifyErrf(w, http.StatusInternalServerError, \"could not get status\")\n\t\treturn\n\t}\n\n\tif logReq.Offset < 0 {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"negative offsets are not supported\")\n\t\treturn\n\t}\n\n\tif status.Index < logReq.Offset {\n\t\tjsonify(w, http.StatusOK, &LogResponse{\n\t\t\tSuccess: true,\n\t\t\tCommits: []Commit{},\n\t\t})\n\t\treturn\n\t}\n\n\tcommits := []Commit{}\n\terrSkip := errors.New(\"skip\")\n\tfilter := strings.ToLower(logReq.Filter)\n\thead := fmt.Sprintf(\"commit[%d]\", -(logReq.Offset + 1))\n\n\terr = lh.fs.Log(head, func(cmt *catfs.Commit) error {\n\t\tif filter != \"\" && !matchCommit(cmt, filter) {\n\t\t\treturn nil\n\t\t}\n\n\t\tif logReq.Limit >= 0 && int64(len(commits)) >= logReq.Limit {\n\t\t\treturn errSkip\n\t\t}\n\n\t\tcommits = append(commits, toExternalCommit(cmt))\n\t\treturn nil\n\t})\n\n\tif err != nil && err != errSkip {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"failed to query log: %v\", err)\n\t\treturn\n\t}\n\n\thaveStagedChanges, err := lh.fs.HaveStagedChanges()\n\tif err != nil {\n\t\tjsonifyErrf(w, http.StatusInternalServerError, \"failed to check staged state: %v\", err)\n\t\treturn\n\t}\n\n\tjsonify(w, http.StatusOK, &LogResponse{\n\t\tSuccess: true,\n\t\tHaveStagedChanges: haveStagedChanges,\n\t\tCommits: commits,\n\t})\n}\n" }, { "path": "gateway/endpoints/log_test.go", "content": "package endpoints\n\nimport (\n\t\"bytes\"\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestLogEndpointSuccess(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\trequire.Nil(t, s.fs.Stage(\"/x\", bytes.NewReader([]byte(\"hello\"))))\n\t\trequire.Nil(t, s.fs.MakeCommit(\"hello\"))\n\t\trequire.Nil(t, s.fs.Stage(\"/x\", bytes.NewReader([]byte(\"world\"))))\n\t\trequire.Nil(t, s.fs.MakeCommit(\"world\"))\n\t\trequire.Nil(t, s.fs.Remove(\"/x\"))\n\t\trequire.Nil(t, s.fs.MakeCommit(\"remove\"))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewLogHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/log\",\n\t\t\t&LogRequest{\n\t\t\t\tOffset: 0,\n\t\t\t\tLimit: -1,\n\t\t\t\tFilter: \"\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tdata := &LogResponse{}\n\t\tmustDecodeBody(t, resp.Body, &data)\n\t\trequire.Equal(t, true, data.Success)\n\t\trequire.Equal(t, 4, len(data.Commits))\n\n\t\trequire.Equal(t, \"\", data.Commits[0].Msg)\n\t\trequire.Equal(t, []string{\"curr\"}, data.Commits[0].Tags)\n\n\t\trequire.Equal(t, \"remove\", data.Commits[1].Msg)\n\t\trequire.Equal(t, []string{\"head\"}, data.Commits[1].Tags)\n\n\t\trequire.Equal(t, \"world\", data.Commits[2].Msg)\n\t\trequire.Equal(t, []string{}, data.Commits[2].Tags)\n\n\t\trequire.Equal(t, \"hello\", data.Commits[3].Msg)\n\t\trequire.Equal(t, []string{\"init\"}, data.Commits[3].Tags)\n\t})\n}\n" }, { "path": "gateway/endpoints/login.go", "content": "package endpoints\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"net/http\"\n\n\t\"github.com/gorilla/sessions\"\n\t\"github.com/sahib/brig/gateway/db\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\nfunc getUserName(store *sessions.CookieStore, w http.ResponseWriter, r *http.Request) string {\n\tsess, err := store.Get(r, \"sess\")\n\tif err != nil {\n\t\tlog.Warningf(\"failed to get session: %v\", err)\n\t\thttp.Error(w, err.Error(), http.StatusInternalServerError)\n\t\treturn \"\"\n\t}\n\n\tuserNameIf, ok := sess.Values[\"name\"]\n\tif !ok {\n\t\treturn \"\"\n\t}\n\n\tuserName, ok := userNameIf.(string)\n\tif !ok {\n\t\tlog.Warningf(\"failed to convert user name to string: %v\", userNameIf)\n\t\thttp.Error(w, \"internal error\", http.StatusInternalServerError)\n\t\treturn \"\"\n\t}\n\n\treturn userName\n}\n\nfunc setSession(store *sessions.CookieStore, userName string, w http.ResponseWriter, r *http.Request) {\n\t// Ignore the error here, since it will usually trigger when there was a previously\n\t// outdated session that fails to decode. Since we overwrite the session anyways, it\n\t// doesn't really matter in this case.\n\tsess, _ := store.Get(r, \"sess\")\n\n\tisHTTPS := r.TLS != nil\n\tsess.Options = &sessions.Options{\n\t\tPath: \"/\",\n\t\tMaxAge: 31 * 24 * 60 * 60,\n\t\tHttpOnly: true,\n\t\tSecure: isHTTPS,\n\t}\n\n\tsess.Values[\"name\"] = userName\n\tif err := sess.Save(r, w); err != nil {\n\t\tlog.Warningf(\"set: failed to save session: %v\", err)\n\t\thttp.Error(w, err.Error(), http.StatusInternalServerError)\n\t\treturn\n\t}\n}\n\nfunc clearSession(store *sessions.CookieStore, w http.ResponseWriter, r *http.Request) {\n\tsess, err := store.Get(r, \"sess\")\n\tif err != nil {\n\t\tlog.Warningf(\"failed to get session: %v\", err)\n\t\thttp.Error(w, err.Error(), http.StatusInternalServerError)\n\t\treturn\n\t}\n\n\tsess.Options.MaxAge = -1\n\tif err := sess.Save(r, w); err != nil {\n\t\tlog.Warningf(\"clear: failed to save session: %v\", err)\n\t\thttp.Error(w, err.Error(), http.StatusInternalServerError)\n\t\treturn\n\t}\n}\n\n///////\n\n// LoginHandler implements http.Handler\ntype LoginHandler struct {\n\t*State\n}\n\n// NewLoginHandler creates a new LoginHandler\nfunc NewLoginHandler(s *State) *LoginHandler {\n\treturn &LoginHandler{State: s}\n}\n\n// LoginRequest is the request sent as JSON to this endpoint.\ntype LoginRequest struct {\n\tUsername string `json:\"username\"`\n\tPassword string `json:\"password\"`\n}\n\n// LoginResponse is what the endpoint will return.\ntype LoginResponse struct {\n\tSuccess bool `json:\"success\"`\n\tUsername string `json:\"username\"`\n\tRights []string `json:\"rights\"`\n\tIsAnon bool `json:\"is_anon\"`\n\tAnonIsAllowed bool `json:\"anon_is_allowed\"`\n}\n\nfunc (lih *LoginHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tloginReq := LoginRequest{}\n\tif err := json.NewDecoder(r.Body).Decode(&loginReq); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"bad json\")\n\t\treturn\n\t}\n\n\tif loginReq.Username == \"\" || loginReq.Password == \"\" {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"empty password or username\")\n\t\treturn\n\t}\n\n\tdbUser, err := lih.userDb.Get(loginReq.Username)\n\tif err != nil {\n\t\t// No such user.\n\t\tjsonifyErrf(w, http.StatusForbidden, \"bad credentials\")\n\t\treturn\n\t}\n\n\tif dbUser.Name != loginReq.Username {\n\t\t// Bad username. Might be a problem on our side.\n\t\tjsonifyErrf(w, http.StatusForbidden, \"bad credentials\")\n\t\treturn\n\t}\n\n\tisValid, err := dbUser.CheckPassword(loginReq.Password)\n\tif err != nil || !isValid {\n\t\tif err != nil {\n\t\t\tlog.Warningf(\"check password failed: %v\", err)\n\t\t}\n\n\t\tjsonifyErrf(w, http.StatusForbidden, \"bad credentials\")\n\t\treturn\n\t}\n\n\tanonIsAllowed := lih.cfg.Bool(\"auth.anon_allowed\")\n\tanonUserName := lih.cfg.String(\"auth.anon_user\")\n\n\tsetSession(lih.store, dbUser.Name, w, r)\n\tjsonify(w, http.StatusOK, &LoginResponse{\n\t\tSuccess: true,\n\t\tUsername: loginReq.Username,\n\t\tRights: dbUser.Rights,\n\t\tIsAnon: anonUserName == loginReq.Username,\n\t\tAnonIsAllowed: anonIsAllowed,\n\t})\n}\n\n///////\n\n// LogoutHandler implements http.Handler\ntype LogoutHandler struct {\n\t*State\n}\n\n// NewLogoutHandler returns a new LogoutHandler\nfunc NewLogoutHandler(s *State) *LogoutHandler {\n\treturn &LogoutHandler{State: s}\n}\n\nfunc (loh *LogoutHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tuser := getUserName(loh.store, w, r)\n\tif user == \"\" {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"not logged in\")\n\t\treturn\n\t}\n\n\tclearSession(loh.store, w, r)\n\tjsonifySuccess(w)\n}\n\n///////\n\n// WhoamiHandler implements http.Handler.\n// This handler checks if a user is already logged in.\ntype WhoamiHandler struct {\n\t*State\n}\n\n// NewWhoamiHandler returns a new WhoamiHandler.\nfunc NewWhoamiHandler(s *State) *WhoamiHandler {\n\treturn &WhoamiHandler{State: s}\n}\n\n// WhoamiResponse is the response sent back by this endpoint.\ntype WhoamiResponse struct {\n\tIsLoggedIn bool `json:\"is_logged_in\"`\n\tIsAnon bool `json:\"is_anon\"`\n\tAnonIsAllowed bool `json:\"anon_is_allowed\"`\n\tUser string `json:\"user\"`\n\tRights []string `json:\"rights\"`\n}\n\nfunc (wh *WhoamiHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\trights := []string{}\n\tisAnon := false\n\n\tanonIsAllowed := wh.cfg.Bool(\"auth.anon_allowed\")\n\tname := getUserName(wh.store, w, r)\n\n\tif name == \"\" && anonIsAllowed {\n\t\tisAnon = true\n\t\tname = wh.cfg.String(\"auth.anon_user\")\n\t}\n\n\tif name != \"\" {\n\t\tpossiblyAnonUser, err := wh.userDb.Get(name)\n\t\tif err != nil {\n\t\t\tlog.Warningf(\"could not get user »%s« : %v\", name, err)\n\t\t} else {\n\t\t\trights = possiblyAnonUser.Rights\n\t\t\tsetSession(wh.store, name, w, r)\n\t\t}\n\t}\n\n\tjsonify(w, http.StatusOK, WhoamiResponse{\n\t\tIsLoggedIn: len(name) > 0,\n\t\tIsAnon: isAnon,\n\t\tAnonIsAllowed: anonIsAllowed,\n\t\tUser: name,\n\t\tRights: rights,\n\t})\n}\n\n///////\n\ntype authMiddleware struct {\n\t*State\n\tSubHandler http.Handler\n}\n\ntype dbUserKey string\n\nfunc (am *authMiddleware) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tanonIsAllowed := am.cfg.Bool(\"auth.anon_allowed\")\n\tname := getUserName(am.store, w, r)\n\n\tif name == \"\" {\n\t\tif !anonIsAllowed {\n\t\t\t// invalid token.\n\t\t\tjsonifyErrf(w, http.StatusUnauthorized, \"not authorized\")\n\t\t\treturn\n\t\t}\n\n\t\tname = am.cfg.String(\"auth.anon_user\")\n\t}\n\n\tuser, err := am.userDb.Get(name)\n\tif err != nil {\n\t\t// valid token, but invalid user.\n\t\t// (user might have been deleted on our side)\n\t\tjsonifyErrf(w, http.StatusUnauthorized, \"not authorized\")\n\t\treturn\n\t}\n\n\tr = r.WithContext(\n\t\tcontext.WithValue(r.Context(), dbUserKey(\"brig.db_user\"),\n\t\t\tuser,\n\t\t),\n\t)\n\n\tam.SubHandler.ServeHTTP(w, r)\n}\n\nfunc checkRights(w http.ResponseWriter, r *http.Request, rights ...string) bool {\n\tuser, ok := r.Context().Value(dbUserKey(\"brig.db_user\")).(db.User)\n\n\tif !ok {\n\t\tjsonifyErrf(w, http.StatusInternalServerError, \"could not cast user\")\n\t\treturn false\n\t}\n\n\trmap := make(map[string]bool)\n\tfor _, right := range user.Rights {\n\t\trmap[right] = true\n\t}\n\n\tfor _, right := range rights {\n\t\tif !rmap[right] {\n\t\t\tjsonifyErrf(w, http.StatusUnauthorized, \"insufficient rights\")\n\t\t\treturn false\n\t\t}\n\t}\n\n\treturn true\n}\n\n// AuthMiddleware returns a new handler wrapper, that will require\n// all calls to the respective handler to have a \"sess\" cookie with\n// a valid user name.\nfunc AuthMiddleware(s *State) func(http.Handler) http.Handler {\n\treturn func(h http.Handler) http.Handler {\n\t\treturn &authMiddleware{State: s, SubHandler: h}\n\t}\n}\n" }, { "path": "gateway/endpoints/login_test.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\ntype loginResponse struct {\n\tSuccess bool `json:\"success\"`\n}\n\nfunc TestLoginEndpointSuccess(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewLoginHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/login\",\n\t\t\t&LoginRequest{\n\t\t\t\tUsername: \"ali\",\n\t\t\t\tPassword: \"ila\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tloginResp := &loginResponse{}\n\t\tmustDecodeBody(t, resp.Body, &loginResp)\n\n\t\trequire.Equal(t, true, loginResp.Success)\n\t\tcookies := resp.Cookies()\n\t\trequire.Equal(t, \"sess\", cookies[0].Name)\n\t})\n}\n" }, { "path": "gateway/endpoints/ls.go", "content": "package endpoints\n\nimport (\n\t\"encoding/json\"\n\t\"net/http\"\n\t\"sort\"\n\t\"strings\"\n\n\t\"github.com/sahib/brig/catfs\"\n\t\"github.com/sahib/brig/gateway/db\"\n)\n\n// LsHandler implements http.Handler.\ntype LsHandler struct {\n\t*State\n}\n\n// NewLsHandler returns a new LsHandler\nfunc NewLsHandler(s *State) *LsHandler {\n\treturn &LsHandler{State: s}\n}\n\n// LsRequest is the data that needs to be sent to this endpoint.\ntype LsRequest struct {\n\tRoot string `json:\"root\"`\n\tFilter string `json:\"filter,omitempty\"`\n}\n\n// StatInfo is a single node in the list response.\n// It is the same as catfs.StatInfo, but is more JSON friendly\n// and omits some fields like hashes that are not useful to the client.\ntype StatInfo struct {\n\tPath string `json:\"path\"`\n\tUser string `json:\"user\"`\n\tSize uint64 `json:\"size\"`\n\tInode uint64 `json:\"inode\"`\n\tDepth int `json:\"depth\"`\n\tModTime int64 `json:\"last_modified_ms\"`\n\tIsDir bool `json:\"is_dir\"`\n\tIsPinned bool `json:\"is_pinned\"`\n\tIsExplicit bool `json:\"is_explicit\"`\n}\n\nfunc toExternalStatInfo(i *catfs.StatInfo) *StatInfo {\n\treturn &StatInfo{\n\t\tPath: i.Path,\n\t\tUser: i.User,\n\t\tSize: i.Size,\n\t\tInode: i.Inode,\n\t\tDepth: i.Depth,\n\t\tModTime: i.ModTime.Unix() * 1000,\n\t\tIsDir: i.IsDir,\n\t\tIsPinned: i.IsPinned,\n\t\tIsExplicit: i.IsExplicit,\n\t}\n}\n\n// LsResponse is the response sent back to the client.\ntype LsResponse struct {\n\tSuccess bool `json:\"success\"`\n\tSelf *StatInfo `json:\"self\"`\n\tFiles []*StatInfo `json:\"files\"`\n\tIsFiltered bool `json:\"is_filtered\"`\n}\n\nfunc doQuery(fs *catfs.FS, root, filter string) ([]*catfs.StatInfo, error) {\n\tif filter == \"\" {\n\t\treturn fs.List(root, 1)\n\t}\n\n\treturn fs.Filter(root, filter)\n}\n\nfunc (lh *LsHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tif !checkRights(w, r, db.RightFsView) {\n\t\treturn\n\t}\n\n\tlsReq := LsRequest{}\n\tif err := json.NewDecoder(r.Body).Decode(&lsReq); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"bad json\")\n\t\treturn\n\t}\n\n\troot := prefixRoot(lsReq.Root)\n\tinfo, err := lh.fs.Stat(root)\n\tif err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"failed to stat root %s: %v\", root, err)\n\t\treturn\n\t}\n\n\titems, err := doQuery(lh.fs, root, lsReq.Filter)\n\tif err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"failed to query: %v\", err)\n\t\treturn\n\t}\n\n\tfiles := []*StatInfo{}\n\tfor _, item := range items {\n\t\tif !lh.pathIsVisible(item.Path, w, r) {\n\t\t\tcontinue\n\t\t}\n\n\t\tfiles = append(files, toExternalStatInfo(item))\n\t}\n\n\t// Sort dirs before files and sort each part alphabetically\n\tsort.Slice(files, func(i, j int) bool {\n\t\tif files[i].IsDir != files[j].IsDir {\n\t\t\treturn files[i].IsDir\n\t\t}\n\n\t\treturn strings.ToLower(files[i].Path) < strings.ToLower(files[j].Path)\n\t})\n\n\tjsonify(w, http.StatusOK, &LsResponse{\n\t\tSuccess: true,\n\t\tFiles: files,\n\t\tIsFiltered: len(lsReq.Filter) > 0,\n\t\tSelf: toExternalStatInfo(info),\n\t})\n}\n" }, { "path": "gateway/endpoints/ls_test.go", "content": "package endpoints\n\nimport (\n\t\"bytes\"\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestLsEndpoint(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\texampleData := bytes.NewReader([]byte(\"Hello world\"))\n\t\trequire.Nil(t, s.fs.Stage(\"/hello/world.png\", exampleData))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewLsHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/ls\",\n\t\t\t&LsRequest{\n\t\t\t\tRoot: \"/\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tlsResp := &LsResponse{}\n\t\tmustDecodeBody(t, resp.Body, &lsResp)\n\n\t\trequire.Len(t, lsResp.Files, 1)\n\t\trequire.Equal(t, lsResp.Files[0].Path, \"/hello\")\n\t})\n}\n" }, { "path": "gateway/endpoints/mkdir.go", "content": "package endpoints\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\n\t\"github.com/sahib/brig/gateway/db\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// MkdirHandler implements http.Handler.\ntype MkdirHandler struct {\n\t*State\n}\n\n// NewMkdirHandler creates a new mkdir handler.\nfunc NewMkdirHandler(s *State) *MkdirHandler {\n\treturn &MkdirHandler{State: s}\n}\n\n// MkdirRequest is the request that can be sent to this endpoint as JSON.\ntype MkdirRequest struct {\n\t// Path to create.\n\tPath string `json:\"path\"`\n}\n\nfunc (mh *MkdirHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tif !checkRights(w, r, db.RightFsEdit) {\n\t\treturn\n\t}\n\n\tmkdirReq := MkdirRequest{}\n\tif err := json.NewDecoder(r.Body).Decode(&mkdirReq); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"bad json\")\n\t\treturn\n\t}\n\n\tpath := prefixRoot(mkdirReq.Path)\n\tif !mh.validatePath(path, w, r) {\n\t\tjsonifyErrf(w, http.StatusUnauthorized, \"path forbidden\")\n\t\treturn\n\t}\n\n\tif err := mh.fs.Mkdir(path, true); err != nil {\n\t\tlog.Debugf(\"failed to mkdir %s: %v\", path, err)\n\t\tjsonifyErrf(w, http.StatusInternalServerError, \"failed to mkdir\")\n\t\treturn\n\t}\n\n\tmsg := fmt.Sprintf(\"mkdir'd »%s«\", path)\n\tif !mh.commitChange(msg, w, r) {\n\t\treturn\n\t}\n\n\tjsonifySuccess(w)\n}\n" }, { "path": "gateway/endpoints/mkdir_test.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\ntype mkdirResponse struct {\n\tSuccess bool `json:\"success\"`\n\tMessage string `json:\"message\"`\n}\n\nfunc TestMkdirEndpointSuccess(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewMkdirHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/mkdir\",\n\t\t\t&MkdirRequest{\n\t\t\t\tPath: \"/test\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tmkdirResp := &mkdirResponse{}\n\t\tmustDecodeBody(t, resp.Body, &mkdirResp)\n\t\trequire.Equal(t, true, mkdirResp.Success)\n\n\t\tinfo, err := s.fs.Stat(\"/test\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, \"/test\", info.Path)\n\t})\n}\n\nfunc TestMkdirEndpointInvalidPath(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\ts.mustChangeFolders(t, \"/something/else\")\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewMkdirHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/mkdir\",\n\t\t\t&MkdirRequest{\n\t\t\t\tPath: \"/test\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusUnauthorized, resp.StatusCode)\n\t})\n}\n" }, { "path": "gateway/endpoints/move.go", "content": "package endpoints\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\n\t\"github.com/sahib/brig/gateway/db\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// MoveHandler implements http.Handler.\ntype MoveHandler struct {\n\t*State\n}\n\n// NewMoveHandler creates a new move handler.\nfunc NewMoveHandler(s *State) *MoveHandler {\n\treturn &MoveHandler{State: s}\n}\n\n// MoveRequest is the request that can be send to this endpoint.\ntype MoveRequest struct {\n\t// Source is the path to the old node.\n\tSource string `json:\"source\"`\n\t// Destination is the path of the new node.\n\tDestination string `json:\"destination\"`\n}\n\nfunc (mh *MoveHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tif !checkRights(w, r, db.RightFsEdit) {\n\t\treturn\n\t}\n\n\tmoveReq := MoveRequest{}\n\tif err := json.NewDecoder(r.Body).Decode(&moveReq); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"bad json\")\n\t\treturn\n\t}\n\n\tsrc := prefixRoot(moveReq.Source)\n\tdst := prefixRoot(moveReq.Destination)\n\n\tif !mh.validatePath(src, w, r) {\n\t\tjsonifyErrf(w, http.StatusUnauthorized, \"source path forbidden\")\n\t\treturn\n\t}\n\n\tif !mh.validatePath(dst, w, r) {\n\t\tjsonifyErrf(w, http.StatusUnauthorized, \"destination path forbidden\")\n\t\treturn\n\t}\n\n\t// Move does some extended checking before actually moving the file:\n\tif err := mh.fs.Move(src, dst); err != nil {\n\t\tlog.Debugf(\"failed to move %s -> %s: %v\", src, dst, err)\n\t\tjsonifyErrf(w, http.StatusInternalServerError, \"failed to move\")\n\t\treturn\n\t}\n\n\tmsg := fmt.Sprintf(\"moved »%s« to »%s« via gateway\", src, dst)\n\tif !mh.commitChange(msg, w, r) {\n\t\treturn\n\t}\n\n\tjsonifySuccess(w)\n}\n" }, { "path": "gateway/endpoints/move_test.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\ntype moveResponse struct {\n\tSuccess bool `json:\"success\"`\n}\n\nfunc TestMoveSuccess(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\trequire.Nil(t, s.fs.Mkdir(\"/hinz\", true))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewMoveHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/move\",\n\t\t\t&MoveRequest{\n\t\t\t\tSource: \"/hinz\",\n\t\t\t\tDestination: \"/kunz\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tmoveResp := &moveResponse{}\n\t\tmustDecodeBody(t, resp.Body, moveResp)\n\t\trequire.Equal(t, true, moveResp.Success)\n\n\t\t_, err := s.fs.Stat(\"/hinz\")\n\t\trequire.NotNil(t, err)\n\n\t\tkunzInfo, err := s.fs.Stat(\"/kunz\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, \"/kunz\", kunzInfo.Path)\n\t})\n}\n\nfunc TestMoveDisallowedSource(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\ts.mustChangeFolders(t, \"/kunz\")\n\t\trequire.Nil(t, s.fs.Mkdir(\"/hinz\", true))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewMoveHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/move\",\n\t\t\t&MoveRequest{\n\t\t\t\tSource: \"/hinz\",\n\t\t\t\tDestination: \"/kunz\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusUnauthorized, resp.StatusCode)\n\t})\n}\n\nfunc TestMoveDisallowedDest(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\ts.mustChangeFolders(t, \"/hinz\")\n\t\trequire.Nil(t, s.fs.Mkdir(\"/hinz\", true))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewMoveHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/move\",\n\t\t\t&MoveRequest{\n\t\t\t\tSource: \"/hinz\",\n\t\t\t\tDestination: \"/kunz\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusUnauthorized, resp.StatusCode)\n\t})\n}\n" }, { "path": "gateway/endpoints/pin.go", "content": "package endpoints\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\n\tie \"github.com/sahib/brig/catfs/errors\"\n\t\"github.com/sahib/brig/gateway/db\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// PinHandler implements http.Handler.\ntype PinHandler struct {\n\t*State\n\tdoPin bool\n}\n\n// NewPinHandler returns a new PinHandler\nfunc NewPinHandler(s *State) *PinHandler {\n\treturn &PinHandler{State: s, doPin: true}\n}\n\n// NewUnpinHandler returns a new PinHandler\nfunc NewUnpinHandler(s *State) *PinHandler {\n\treturn &PinHandler{State: s, doPin: false}\n}\n\n// PinRequest is the request that is being sent to the endpoint.\ntype PinRequest struct {\n\tPath string `json:\"path\"`\n\tRevision string `json:\"revision\"`\n\tDoPin bool `json:\"do_pin\"`\n}\n\nfunc (ph *PinHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tif !checkRights(w, r, db.RightFsEdit) {\n\t\treturn\n\t}\n\n\tpinReq := PinRequest{}\n\tif err := json.NewDecoder(r.Body).Decode(&pinReq); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"bad json\")\n\t\treturn\n\t}\n\n\tpath := prefixRoot(pinReq.Path)\n\tif !ph.validatePath(path, w, r) {\n\t\tjsonifyErrf(w, http.StatusUnauthorized, \"path forbidden\")\n\t\treturn\n\t}\n\n\t// Select the right operation:\n\top, name := ph.fs.Pin, \"pin\"\n\tif ph.doPin == false {\n\t\top, name = ph.fs.Unpin, \"unpin\"\n\t}\n\n\tif err := op(path, pinReq.Revision, true); err != nil {\n\t\tif !ie.IsNoSuchFileError(err) {\n\t\t\tlog.Debugf(\"failed to %s %s: %v\", name, path, err)\n\t\t\tjsonifyErrf(w, http.StatusBadRequest, fmt.Sprintf(\"failed to %s\", name))\n\t\t\treturn\n\t\t}\n\t}\n\n\tph.evHdl.Notify(r.Context(), \"pin\")\n\tjsonifySuccess(w)\n}\n" }, { "path": "gateway/endpoints/pin_test.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\ntype pinResponse struct {\n\tSuccess bool `json:\"success\"`\n\tMessage string `json:\"message\"`\n}\n\nfunc TestPinEndpointSuccess(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\trequire.Nil(t, s.fs.Touch(\"/file\"))\n\t\trequire.Nil(t, s.fs.Mkdir(\"/dir\", true))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewPinHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/pin\",\n\t\t\t&PinRequest{\n\t\t\t\tPath: \"/file\",\n\t\t\t\tRevision: \"curr\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tpinResp := &pinResponse{}\n\t\tmustDecodeBody(t, resp.Body, &pinResp)\n\t\trequire.Equal(t, true, pinResp.Success)\n\n\t\tstat, err := s.fs.Stat(\"/file\")\n\t\trequire.Nil(t, err)\n\t\trequire.True(t, stat.IsPinned)\n\t\trequire.True(t, stat.IsExplicit)\n\n\t\tstat, err = s.fs.Stat(\"/dir\")\n\t\trequire.Nil(t, err)\n\t\trequire.True(t, stat.IsPinned)\n\t\trequire.True(t, stat.IsExplicit)\n\t})\n}\n\nfunc TestPinEndpointForbidden(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\ts.mustChangeFolders(t, \"/public\")\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewPinHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/pin\",\n\t\t\t&PinRequest{\n\t\t\t\tPath: \"/file\",\n\t\t\t\tRevision: \"curr\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusUnauthorized, resp.StatusCode)\n\t\tpinResp := &pinResponse{}\n\t\tmustDecodeBody(t, resp.Body, &pinResp)\n\t\trequire.Equal(t, false, pinResp.Success)\n\t})\n}\n" }, { "path": "gateway/endpoints/ping.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n)\n\n// PingHandler implements http.Handler.\n// This handler checks if a user is already logged in.\ntype PingHandler struct {\n\t*State\n}\n\n// NewPingHandler returns a new PingHandler.\nfunc NewPingHandler(s *State) *PingHandler {\n\treturn &PingHandler{State: s}\n}\n\n// PingResponse is the response sent back by this endpoint.\ntype PingResponse struct {\n\tIsOnline bool `json:\"is_online\"`\n}\n\nfunc (wh *PingHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tjsonify(w, http.StatusOK, PingResponse{\n\t\tIsOnline: true,\n\t})\n}\n" }, { "path": "gateway/endpoints/ping_test.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestPingEndpointSuccess(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewPingHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/ping\",\n\t\t\tnil,\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\t\tpingResp := &PingResponse{}\n\t\tmustDecodeBody(t, resp.Body, &pingResp)\n\t\trequire.Equal(t, true, pingResp.IsOnline)\n\t})\n}\n" }, { "path": "gateway/endpoints/redirect.go", "content": "package endpoints\n\nimport (\n\t\"fmt\"\n\t\"net\"\n\t\"net/http\"\n\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// RedirHandler implements http.Handler.\n// It redirects all of its requests to the respective https:// route.\ntype RedirHandler struct {\n\tredirPort int64\n}\n\n// NewHTTPRedirectHandler returns a new RedirHandler\nfunc NewHTTPRedirectHandler(redirPort int64) *RedirHandler {\n\treturn &RedirHandler{\n\t\tredirPort: redirPort,\n\t}\n}\n\nfunc (rh *RedirHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {\n\t// remove/add not default ports from req.Host\n\thost, _, err := net.SplitHostPort(req.Host)\n\tif err != nil {\n\t\thttp.Error(w, \"invalid request\", http.StatusBadRequest)\n\t\treturn\n\t}\n\n\ttarget := fmt.Sprintf(\"https://%s:%d%s\", host, rh.redirPort, req.URL.Path)\n\tif len(req.URL.RawQuery) > 0 {\n\t\ttarget += \"?\" + req.URL.RawQuery\n\t}\n\n\tlog.Debugf(\"redirect to: %s\", target)\n\thttp.Redirect(w, req, target, http.StatusTemporaryRedirect)\n}\n" }, { "path": "gateway/endpoints/remotes_add.go", "content": "package endpoints\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"strings\"\n\n\t\"github.com/sahib/brig/gateway/db\"\n\t\"github.com/sahib/brig/gateway/remotesapi\"\n\t\"github.com/sahib/brig/net/peer\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// RemotesAddHandler implements http.Handler\ntype RemotesAddHandler struct {\n\t*State\n}\n\n// NewRemotesAddHandler returns a new RemotesAddHandler\nfunc NewRemotesAddHandler(s *State) *RemotesAddHandler {\n\treturn &RemotesAddHandler{State: s}\n}\n\n// RemoteAddRequest is the data being sent to this endpoint.\ntype RemoteAddRequest struct {\n\tName string `json:\"name\"`\n\tFolders []remotesapi.Folder `json:\"folders\"`\n\tFingerprint string `json:\"fingerprint\"`\n\tAcceptAutoUpdates bool `json:\"accept_auto_updates\"`\n\tAcceptPush bool `json:\"accept_push\"`\n\tConflictStrategy string `json:\"conflict_strategy\"`\n}\n\nfunc dedupeFolders(folders []remotesapi.Folder) []remotesapi.Folder {\n\tseen := make(map[string]bool)\n\tdeduped := []remotesapi.Folder{}\n\n\tfor _, folder := range folders {\n\t\tpath := folder.Folder\n\t\tif !strings.HasPrefix(path, \"/\") {\n\t\t\tpath = \"/\" + path\n\t\t}\n\n\t\tif seen[path] {\n\t\t\tcontinue\n\t\t}\n\n\t\tdeduped = append(deduped, folder)\n\t\tseen[path] = true\n\t}\n\n\treturn deduped\n}\n\nfunc validateFingerprint(fingerprint string, w http.ResponseWriter, r *http.Request) bool {\n\tif _, err := peer.CastFingerprint(fingerprint); err != nil {\n\t\tlog.Debugf(\"invalid fingerprint: %v\", err)\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"bad fingerprint format\")\n\t\treturn false\n\t}\n\n\treturn true\n}\n\nfunc readRemoteRequest(w http.ResponseWriter, r *http.Request) (*remotesapi.Remote, error) {\n\tif !checkRights(w, r, db.RightRemotesEdit) {\n\t\treturn nil, fmt.Errorf(\"bad rights\")\n\t}\n\n\tremoteAddReq := RemoteAddRequest{}\n\tif err := json.NewDecoder(r.Body).Decode(&remoteAddReq); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"bad json\")\n\t\treturn nil, fmt.Errorf(\"bad json\")\n\t}\n\n\tif !validateFingerprint(remoteAddReq.Fingerprint, w, r) {\n\t\treturn nil, fmt.Errorf(\"bad fingerprint\")\n\t}\n\n\treturn &remotesapi.Remote{\n\t\tName: remoteAddReq.Name,\n\t\tFolders: dedupeFolders(remoteAddReq.Folders),\n\t\tFingerprint: remoteAddReq.Fingerprint,\n\t\tAcceptAutoUpdates: remoteAddReq.AcceptAutoUpdates,\n\t\tAcceptPush: remoteAddReq.AcceptPush,\n\t\tConflictStrategy: remoteAddReq.ConflictStrategy,\n\t}, nil\n}\n\nfunc (rh *RemotesAddHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\trmt, err := readRemoteRequest(w, r)\n\tif err != nil {\n\t\treturn\n\t}\n\n\tif _, err := rh.rapi.Get(rmt.Name); err == nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"remote does exist already\")\n\t\treturn\n\t}\n\n\tif err := rh.rapi.Set(*rmt); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"failed to add\")\n\t\treturn\n\t}\n\n\tjsonifySuccess(w)\n}\n\n//////////////\n\n// RemotesModifyHandler implements http.Handler\ntype RemotesModifyHandler struct {\n\t*State\n}\n\n// NewRemotesModifyHandler returns a new RemotesModifyHandler\nfunc NewRemotesModifyHandler(s *State) *RemotesModifyHandler {\n\treturn &RemotesModifyHandler{State: s}\n}\n\nfunc (rh *RemotesModifyHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\trmt, err := readRemoteRequest(w, r)\n\tif err != nil {\n\t\treturn\n\t}\n\n\tif _, err := rh.rapi.Get(rmt.Name); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"remote does not exist yet\")\n\t\treturn\n\t}\n\n\tif err := rh.rapi.Set(*rmt); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"failed to add\")\n\t\treturn\n\t}\n\n\tjsonifySuccess(w)\n}\n" }, { "path": "gateway/endpoints/remotes_add_test.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/sahib/brig/gateway/remotesapi\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nconst (\n\tTestFingerprint = \"QmgtEcRda8Nm4RMHQCBzGGXBE2zjQqvDXHfEye1zay3f1w:W1fKKbqVAUhEXkC3yoJ92fKK1aWAuVYMmneUUvUQdDRbMq\"\n)\n\nfunc TestRemoteAddEndpoint(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewRemotesAddHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/remotes/add\",\n\t\t\t&RemoteAddRequest{\n\t\t\t\tName: \"bob\",\n\t\t\t\tFolders: nil,\n\t\t\t\tFingerprint: TestFingerprint,\n\t\t\t\tAcceptAutoUpdates: true,\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tdata := struct {\n\t\t\tSuccess bool `json:\"success\"`\n\t\t}{}\n\t\tmustDecodeBody(t, resp.Body, &data)\n\t\trequire.Equal(t, true, data.Success)\n\n\t\trmt, err := s.State.rapi.Get(\"bob\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, \"bob\", rmt.Name)\n\t\trequire.Equal(t, TestFingerprint, rmt.Fingerprint)\n\t\trequire.Equal(t, true, rmt.AcceptAutoUpdates)\n\t})\n}\n\nfunc TestRemoteModifyEndpoint(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\trequire.Nil(t, s.State.rapi.Set(remotesapi.Remote{\n\t\t\tName: \"bob\",\n\t\t\tFingerprint: TestFingerprint + \"xxx\",\n\t\t\tFolders: []remotesapi.Folder{\n\t\t\t\t{\n\t\t\t\t\tFolder: \"/public\",\n\t\t\t\t\tReadOnly: false,\n\t\t\t\t\tConflictStrategy: \"\",\n\t\t\t\t},\n\t\t\t},\n\t\t}))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewRemotesModifyHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/remotes/modify\",\n\t\t\t&RemoteAddRequest{\n\t\t\t\tName: \"bob\",\n\t\t\t\tFolders: nil,\n\t\t\t\tFingerprint: TestFingerprint,\n\t\t\t\tAcceptAutoUpdates: true,\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tdata := struct {\n\t\t\tSuccess bool `json:\"success\"`\n\t\t}{}\n\n\t\tmustDecodeBody(t, resp.Body, &data)\n\t\trequire.Equal(t, true, data.Success)\n\n\t\trmt, err := s.State.rapi.Get(\"bob\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, \"bob\", rmt.Name)\n\t\trequire.Equal(t, TestFingerprint, rmt.Fingerprint)\n\t\trequire.Equal(t, true, rmt.AcceptAutoUpdates)\n\t})\n}\n" }, { "path": "gateway/endpoints/remotes_diff.go", "content": "package endpoints\n\nimport (\n\t\"encoding/json\"\n\t\"net/http\"\n\n\t\"github.com/sahib/brig/catfs\"\n\t\"github.com/sahib/brig/gateway/db\"\n)\n\n// RemotesDiffHandler implements http.Handler\ntype RemotesDiffHandler struct {\n\t*State\n}\n\n// NewRemotesDiffHandler returns a new RemotesDiffHandler\nfunc NewRemotesDiffHandler(s *State) *RemotesDiffHandler {\n\treturn &RemotesDiffHandler{State: s}\n}\n\n// RemoteDiffRequest is the data being sent to this endpoint.\ntype RemoteDiffRequest struct {\n\tName string `json:\"name\"`\n}\n\n// DiffPair is like catfs.DiffPair, but with some\n// fields removed and with json instructions.\ntype DiffPair struct {\n\tSrc *StatInfo `json:\"src\"`\n\tDst *StatInfo `json:\"dst\"`\n}\n\n// Diff is like catfs.Diff, but json-ized.\ntype Diff struct {\n\tAdded []*StatInfo `json:\"added\"`\n\tRemoved []*StatInfo `json:\"removed\"`\n\tIgnored []*StatInfo `json:\"ignored\"`\n\tMissing []*StatInfo `json:\"missing\"`\n\tConflict []DiffPair `json:\"conflict\"`\n\tMoved []DiffPair `json:\"moved\"`\n\tMerged []DiffPair `json:\"merged\"`\n}\n\n// RemoteDiffResponse is the data being sent to this endpoint.\ntype RemoteDiffResponse struct {\n\tSuccess bool `json:\"success\"`\n\tDiff *Diff `json:\"diff\"`\n}\n\nfunc convertSingles(infos []catfs.StatInfo) []*StatInfo {\n\tresult := []*StatInfo{}\n\tfor _, info := range infos {\n\t\tresult = append(result, toExternalStatInfo(&info))\n\t}\n\n\treturn result\n}\n\nfunc convertPairs(pairs []catfs.DiffPair) []DiffPair {\n\tresult := []DiffPair{}\n\tfor _, pair := range pairs {\n\t\tresult = append(result, DiffPair{\n\t\t\tSrc: toExternalStatInfo(&pair.Src),\n\t\t\tDst: toExternalStatInfo(&pair.Dst),\n\t\t})\n\t}\n\n\treturn result\n\n}\n\nfunc (rh *RemotesDiffHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tif !checkRights(w, r, db.RightRemotesView) {\n\t\treturn\n\t}\n\n\trmtDiffReq := RemoteDiffRequest{}\n\tif err := json.NewDecoder(r.Body).Decode(&rmtDiffReq); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"bad json\")\n\t\treturn\n\t}\n\n\tif rmtDiffReq.Name == \"\" {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"empty remote name\")\n\t\treturn\n\t}\n\n\trawDiff, err := rh.rapi.MakeDiff(rmtDiffReq.Name)\n\tif err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"failed to diff\")\n\t\treturn\n\t}\n\n\tdiff := &Diff{\n\t\tAdded: convertSingles(rawDiff.Added),\n\t\tRemoved: convertSingles(rawDiff.Removed),\n\t\tIgnored: convertSingles(rawDiff.Ignored),\n\t\tMissing: convertSingles(rawDiff.Missing),\n\t\tConflict: convertPairs(rawDiff.Conflict),\n\t\tMoved: convertPairs(rawDiff.Moved),\n\t\tMerged: convertPairs(rawDiff.Merged),\n\t}\n\n\tjsonify(w, http.StatusOK, RemoteDiffResponse{\n\t\tSuccess: true,\n\t\tDiff: diff,\n\t})\n}\n" }, { "path": "gateway/endpoints/remotes_diff_test.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/sahib/brig/gateway/remotesapi\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestRemoteDiffEndpoint(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\trequire.Nil(t, s.State.rapi.Set(remotesapi.Remote{\n\t\t\tName: \"bob\",\n\t\t\tFingerprint: \"xxx\",\n\t\t}))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewRemotesDiffHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/remotes/diff\",\n\t\t\tRemoteDiffRequest{\n\t\t\t\tName: \"bob\",\n\t\t\t},\n\t\t)\n\n\t\tdata := &RemoteDiffResponse{}\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\t\tmustDecodeBody(t, resp.Body, &data)\n\t\trequire.Equal(t, true, data.Success)\n\n\t\trequire.Equal(t, 2, len(data.Diff.Added))\n\t\trequire.Equal(t, \"/new_dir\", data.Diff.Added[0].Path)\n\t\trequire.Equal(t, \"/new_file\", data.Diff.Added[1].Path)\n\n\t\trequire.Equal(t, 1, len(data.Diff.Removed))\n\t\trequire.Equal(t, \"/removed_file\", data.Diff.Removed[0].Path)\n\n\t\trequire.Equal(t, 1, len(data.Diff.Ignored))\n\t\trequire.Equal(t, \"/ignored\", data.Diff.Ignored[0].Path)\n\n\t\trequire.Equal(t, 1, len(data.Diff.Missing))\n\t\trequire.Equal(t, \"/missing\", data.Diff.Missing[0].Path)\n\n\t\trequire.Equal(t, 1, len(data.Diff.Conflict))\n\t\trequire.Equal(t, \"/conflict_src\", data.Diff.Conflict[0].Src.Path)\n\t\trequire.Equal(t, \"/conflict_dst\", data.Diff.Conflict[0].Dst.Path)\n\n\t\trequire.Equal(t, 1, len(data.Diff.Moved))\n\t\trequire.Equal(t, \"/moved_src\", data.Diff.Moved[0].Src.Path)\n\t\trequire.Equal(t, \"/moved_dst\", data.Diff.Moved[0].Dst.Path)\n\n\t\trequire.Equal(t, 1, len(data.Diff.Merged))\n\t\trequire.Equal(t, \"/merged_src\", data.Diff.Merged[0].Src.Path)\n\t\trequire.Equal(t, \"/merged_dst\", data.Diff.Merged[0].Dst.Path)\n\t})\n}\n" }, { "path": "gateway/endpoints/remotes_list.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n\t\"sort\"\n\n\t\"github.com/sahib/brig/gateway/db\"\n\t\"github.com/sahib/brig/gateway/remotesapi\"\n)\n\n// RemoteListHandler implements http.Handler\ntype RemoteListHandler struct {\n\t*State\n}\n\n// NewRemotesListHandler returns a new RemoteListHandler\nfunc NewRemotesListHandler(s *State) *RemoteListHandler {\n\treturn &RemoteListHandler{State: s}\n}\n\n// RemoteListResponse is the response given by this endpoint.\ntype RemoteListResponse struct {\n\tSuccess bool `json:\"success\"`\n\tRemotes []*remotesapi.Remote `json:\"remotes\"`\n}\n\nfunc (rh *RemoteListHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tif !checkRights(w, r, db.RightRemotesView) {\n\t\treturn\n\t}\n\n\trmts, err := rh.rapi.List()\n\tif err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"bad json\")\n\t\treturn\n\t}\n\n\tsort.Slice(rmts, func(i, j int) bool {\n\t\treturn rmts[i].Name < rmts[j].Name\n\t})\n\n\tjsonify(w, http.StatusOK, &RemoteListResponse{\n\t\tSuccess: true,\n\t\tRemotes: rmts,\n\t})\n}\n" }, { "path": "gateway/endpoints/remotes_list_test.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/sahib/brig/gateway/remotesapi\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestRemoteListEndpoint(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\trequire.Nil(t, s.State.rapi.Set(remotesapi.Remote{\n\t\t\tName: \"bob\",\n\t\t\tFingerprint: \"xxx\",\n\t\t}))\n\n\t\trequire.Nil(t, s.State.rapi.Set(remotesapi.Remote{\n\t\t\tName: \"charlie\",\n\t\t\tFingerprint: \"yyy\",\n\t\t\tAcceptAutoUpdates: true,\n\t\t\tFolders: []remotesapi.Folder{\n\t\t\t\t{\n\t\t\t\t\tFolder: \"/public\",\n\t\t\t\t\tReadOnly: false,\n\t\t\t\t\tConflictStrategy: \"\",\n\t\t\t\t},\n\t\t\t},\n\t\t}))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewRemotesListHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/remotes/list\",\n\t\t\tnil,\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tdata := &RemoteListResponse{}\n\t\tmustDecodeBody(t, resp.Body, &data)\n\n\t\trequire.Equal(t, true, data.Success)\n\t\trequire.Equal(t, 2, len(data.Remotes))\n\t\trequire.Equal(t, \"bob\", data.Remotes[0].Name)\n\t\trequire.Equal(t, \"xxx\", data.Remotes[0].Fingerprint)\n\t\trequire.Equal(t, false, data.Remotes[0].AcceptAutoUpdates)\n\n\t\trequire.Equal(t, \"charlie\", data.Remotes[1].Name)\n\t\trequire.Equal(t, \"yyy\", data.Remotes[1].Fingerprint)\n\t\trequire.Equal(t, true, data.Remotes[1].AcceptAutoUpdates)\n\t})\n}\n" }, { "path": "gateway/endpoints/remotes_remove.go", "content": "package endpoints\n\nimport (\n\t\"encoding/json\"\n\t\"net/http\"\n\n\t\"github.com/sahib/brig/gateway/db\"\n)\n\n// RemotesRemoveHandler implements http.Handler\ntype RemotesRemoveHandler struct {\n\t*State\n}\n\n// NewRemotesRemoveHandler returns a new RemotesRemoveHandler\nfunc NewRemotesRemoveHandler(s *State) *RemotesRemoveHandler {\n\treturn &RemotesRemoveHandler{State: s}\n}\n\n// RemoteRemoveRequest is the data being sent to this endpoint.\ntype RemoteRemoveRequest struct {\n\tName string `json:\"name\"`\n}\n\nfunc (rh *RemotesRemoveHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tif !checkRights(w, r, db.RightRemotesEdit) {\n\t\treturn\n\t}\n\n\trmtRmReq := RemoteRemoveRequest{}\n\tif err := json.NewDecoder(r.Body).Decode(&rmtRmReq); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"bad json\")\n\t\treturn\n\t}\n\n\tif rmtRmReq.Name == \"\" {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"empty remote name\")\n\t\treturn\n\t}\n\n\tif err := rh.rapi.Remove(rmtRmReq.Name); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"failed to remove remote\")\n\t\treturn\n\t}\n\n\tjsonifySuccess(w)\n}\n" }, { "path": "gateway/endpoints/remotes_remove_test.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/sahib/brig/gateway/remotesapi\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestRemoteRemoveEndpoint(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\trequire.Nil(t, s.State.rapi.Set(remotesapi.Remote{\n\t\t\tName: \"bob\",\n\t\t\tFingerprint: \"xxx\",\n\t\t}))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewRemotesRemoveHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/remotes/remove\",\n\t\t\tRemoteRemoveRequest{\n\t\t\t\tName: \"bob\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\t\tdata := struct {\n\t\t\tSuccess bool `json:\"success\"`\n\t\t}{}\n\t\tmustDecodeBody(t, resp.Body, &data)\n\t\trequire.Equal(t, true, data.Success)\n\n\t\tresp = s.mustRun(\n\t\t\tt,\n\t\t\tNewRemotesRemoveHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/remotes/remove\",\n\t\t\tRemoteRemoveRequest{\n\t\t\t\tName: \"bob\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusBadRequest, resp.StatusCode)\n\t})\n}\n" }, { "path": "gateway/endpoints/remotes_self.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n\n\t\"github.com/sahib/brig/gateway/db\"\n\t\"github.com/sahib/brig/gateway/remotesapi\"\n)\n\n// RemoteSelfHandler implements http.Handler\ntype RemoteSelfHandler struct {\n\t*State\n}\n\n// NewRemotesSelfHandler returns a new RemoteSelfHandler\nfunc NewRemotesSelfHandler(s *State) *RemoteSelfHandler {\n\treturn &RemoteSelfHandler{State: s}\n}\n\n// RemoteSelfResponse is the data being sent to this endpoint.\ntype RemoteSelfResponse struct {\n\tSuccess bool `json:\"success\"`\n\tSelf remotesapi.Identity `json:\"self\"`\n\tDefaultConflictStrategy string `json:\"default_conflict_strategy\"`\n}\n\nfunc (rh *RemoteSelfHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tif !checkRights(w, r, db.RightRemotesView) {\n\t\treturn\n\t}\n\n\tself, err := rh.rapi.Self()\n\tif err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"failed to get self\")\n\t\treturn\n\t}\n\n\tjsonify(w, http.StatusOK, RemoteSelfResponse{\n\t\tSuccess: true,\n\t\tSelf: self,\n\t\tDefaultConflictStrategy: \"marker\",\n\t})\n}\n" }, { "path": "gateway/endpoints/remotes_self_test.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestRemoteSelfEndpoint(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewRemotesSelfHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/remotes/self\",\n\t\t\tnil,\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tdata := &RemoteSelfResponse{}\n\t\tmustDecodeBody(t, resp.Body, &data)\n\t\trequire.Equal(t, true, data.Success)\n\t\trequire.Equal(t, \"ali\", data.Self.Name)\n\t\trequire.Equal(t, \"alisfingerprint\", data.Self.Fingerprint)\n\t})\n}\n" }, { "path": "gateway/endpoints/remotes_sync.go", "content": "package endpoints\n\nimport (\n\t\"encoding/json\"\n\t\"net/http\"\n\n\t\"github.com/sahib/brig/gateway/db\"\n)\n\n// RemotesSyncHandler implements http.Handler\ntype RemotesSyncHandler struct {\n\t*State\n}\n\n// NewRemotesSyncHandler returns a new RemotesSyncHandler\nfunc NewRemotesSyncHandler(s *State) *RemotesSyncHandler {\n\treturn &RemotesSyncHandler{State: s}\n}\n\n// RemoteSyncRequest is the data being sent to this endpoint.\ntype RemoteSyncRequest struct {\n\tName string `json:\"name\"`\n}\n\nfunc (rh *RemotesSyncHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tif !checkRights(w, r, db.RightRemotesEdit, db.RightFsEdit) {\n\t\treturn\n\t}\n\n\trmtSyncReq := RemoteSyncRequest{}\n\tif err := json.NewDecoder(r.Body).Decode(&rmtSyncReq); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"bad json\")\n\t\treturn\n\t}\n\n\tif rmtSyncReq.Name == \"\" {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"empty remote name\")\n\t\treturn\n\t}\n\n\tif err := rh.rapi.Sync(rmtSyncReq.Name); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"failed to sync\")\n\t\treturn\n\t}\n\n\tjsonifySuccess(w)\n}\n" }, { "path": "gateway/endpoints/remotes_sync_test.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/sahib/brig/gateway/remotesapi\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestRemoteSyncEndpoint(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\trequire.Nil(t, s.State.rapi.Set(remotesapi.Remote{\n\t\t\tName: \"bob\",\n\t\t\tFingerprint: \"xxx\",\n\t\t}))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewRemotesSyncHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/remotes/sync\",\n\t\t\tRemoteSyncRequest{\n\t\t\t\tName: \"bob\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\t\tdata := struct {\n\t\t\tSuccess bool `json:\"success\"`\n\t\t}{}\n\t\tmustDecodeBody(t, resp.Body, &data)\n\t\trequire.Equal(t, true, data.Success)\n\t})\n}\n" }, { "path": "gateway/endpoints/remove.go", "content": "package endpoints\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\n\t\"github.com/sahib/brig/gateway/db\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// RemoveHandler implements http.Handler.\ntype RemoveHandler struct {\n\t*State\n}\n\n// NewRemoveHandler returns a new RemoveHandler\nfunc NewRemoveHandler(s *State) *RemoveHandler {\n\treturn &RemoveHandler{State: s}\n}\n\n// RemoveRequest is the request that is being sent to the endpoint.\ntype RemoveRequest struct {\n\tPaths []string `json:\"paths\"`\n}\n\nfunc (rh *RemoveHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tif !checkRights(w, r, db.RightFsEdit) {\n\t\treturn\n\t}\n\n\trmReq := RemoveRequest{}\n\tif err := json.NewDecoder(r.Body).Decode(&rmReq); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"bad json\")\n\t\treturn\n\t}\n\n\tfor _, path := range rmReq.Paths {\n\t\tpath = prefixRoot(path)\n\t\tif !rh.validatePath(path, w, r) {\n\t\t\tjsonifyErrf(w, http.StatusUnauthorized, \"path forbidden\")\n\t\t\treturn\n\t\t}\n\t}\n\n\tpaths := []string{}\n\tfor _, path := range rmReq.Paths {\n\t\tpath = prefixRoot(path)\n\t\tif err := rh.fs.Remove(path); err != nil {\n\t\t\tlog.Debugf(\"failed to remove %s: %v\", path, err)\n\t\t\tjsonifyErrf(w, http.StatusBadRequest, \"failed to remove\")\n\t\t\treturn\n\t\t}\n\n\t\tpaths = append(paths, path)\n\t}\n\n\tif len(paths) > 0 {\n\t\tmsg := fmt.Sprintf(\"removed »%s«\", paths[0])\n\t\tif len(paths) > 1 {\n\t\t\tmsg += fmt.Sprintf(\" and %d others\", len(paths)-1)\n\t\t}\n\n\t\tif !rh.commitChange(msg, w, r) {\n\t\t\treturn\n\t\t}\n\t}\n\n\tjsonifySuccess(w)\n}\n" }, { "path": "gateway/endpoints/remove_test.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\ntype removeResponse struct {\n\tSuccess bool `json:\"success\"`\n\tMessage string `json:\"message\"`\n}\n\nfunc TestRemoveEndpointSuccess(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\trequire.Nil(t, s.fs.Touch(\"/file\"))\n\t\trequire.Nil(t, s.fs.Mkdir(\"/dir\", true))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewRemoveHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/remove\",\n\t\t\t&RemoveRequest{\n\t\t\t\tPaths: []string{\"/file\", \"/dir\"},\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tremoveResp := &removeResponse{}\n\t\tmustDecodeBody(t, resp.Body, &removeResp)\n\t\trequire.Equal(t, true, removeResp.Success)\n\n\t\t_, err := s.fs.Stat(\"/file\")\n\t\trequire.NotNil(t, err)\n\n\t\t_, err = s.fs.Stat(\"/dir\")\n\t\trequire.NotNil(t, err)\n\t})\n}\n\nfunc TestRemoveEndpointForbidden(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\ts.mustChangeFolders(t, \"/public\")\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewRemoveHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/remove\",\n\t\t\t&RemoveRequest{\n\t\t\t\tPaths: []string{\"/file\", \"/dir\"},\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusUnauthorized, resp.StatusCode)\n\t\tremoveResp := &removeResponse{}\n\t\tmustDecodeBody(t, resp.Body, &removeResp)\n\t\trequire.Equal(t, false, removeResp.Success)\n\t})\n}\n" }, { "path": "gateway/endpoints/reset.go", "content": "package endpoints\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\n\t\"github.com/sahib/brig/gateway/db\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// ResetHandler implements http.Handler.\ntype ResetHandler struct {\n\t*State\n}\n\n// NewResetHandler returns a new ResetHandler.\nfunc NewResetHandler(s *State) *ResetHandler {\n\treturn &ResetHandler{State: s}\n}\n\n// ResetRequest is a request sent to this endpoint.\ntype ResetRequest struct {\n\tPath string `json:\"path\"`\n\tRevision string `json:\"revision\"`\n\tForce bool `json:\"force\"`\n}\n\nfunc (rh *ResetHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tif !checkRights(w, r, db.RightFsEdit) {\n\t\treturn\n\t}\n\n\tresetReq := ResetRequest{}\n\tif err := json.NewDecoder(r.Body).Decode(&resetReq); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"bad json\")\n\t\treturn\n\t}\n\n\tpath := prefixRoot(resetReq.Path)\n\tif !rh.validatePath(path, w, r) {\n\t\tjsonifyErrf(w, http.StatusUnauthorized, \"path forbidden\")\n\t\treturn\n\t}\n\n\tvar err error\n\tif resetReq.Path == \"/\" {\n\t\terr = rh.fs.Checkout(resetReq.Revision, true)\n\t} else {\n\t\terr = rh.fs.Reset(path, resetReq.Revision)\n\t}\n\n\tlog.Debugf(\"reset %s to %s\", path, resetReq.Revision)\n\tif err != nil {\n\t\tlog.Debugf(\"failed to reset %s to %s: %v\", path, resetReq.Revision, err)\n\t\tjsonifyErrf(w, http.StatusInternalServerError, \"failed to reset\")\n\t\treturn\n\t}\n\n\tmsg := fmt.Sprintf(\"reverted »%s« to »%s«\", path, resetReq.Revision)\n\tif !rh.commitChange(msg, w, r) {\n\t\treturn\n\t}\n\n\tjsonifySuccess(w)\n}\n" }, { "path": "gateway/endpoints/reset_test.go", "content": "package endpoints\n\nimport (\n\t\"bytes\"\n\t\"io/ioutil\"\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\ntype resetResponse struct {\n\tSuccess bool `json:\"success\"`\n\tMessage string `json:\"message\"`\n}\n\nfunc TestResetSuccess(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\trequire.Nil(t, s.fs.Stage(\"/file\", bytes.NewReader([]byte(\"hello\"))))\n\t\trequire.Nil(t, s.fs.MakeCommit(\"add\"))\n\t\trequire.Nil(t, s.fs.Stage(\"/file\", bytes.NewReader([]byte(\"world\"))))\n\t\trequire.Nil(t, s.fs.MakeCommit(\"modify\"))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewResetHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/reset\",\n\t\t\t&ResetRequest{\n\t\t\t\tPath: \"/file\",\n\t\t\t\tRevision: \"init\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tresetResp := &resetResponse{}\n\t\tmustDecodeBody(t, resp.Body, &resetResp)\n\t\trequire.Equal(t, true, resetResp.Success)\n\n\t\tstream, err := s.fs.Cat(\"/file\")\n\t\trequire.Nil(t, err)\n\n\t\tdata, err := ioutil.ReadAll(stream)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, []byte(\"hello\"), data)\n\t})\n}\n\nfunc TestResetForbidden(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\ts.mustChangeFolders(t, \"/public\")\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewResetHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/reset\",\n\t\t\t&ResetRequest{\n\t\t\t\tPath: \"/file\",\n\t\t\t\tRevision: \"init\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusUnauthorized, resp.StatusCode)\n\n\t\tresetResp := &resetResponse{}\n\t\tmustDecodeBody(t, resp.Body, &resetResp)\n\t\trequire.Equal(t, false, resetResp.Success)\n\t})\n}\n" }, { "path": "gateway/endpoints/testing.go", "content": "package endpoints\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\n\t\"github.com/sahib/brig/catfs\"\n\t\"github.com/sahib/brig/defaults\"\n\t\"github.com/sahib/brig/gateway/db\"\n\t\"github.com/sahib/brig/gateway/remotesapi\"\n\t\"github.com/sahib/config\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nconst (\n\ttestGwUser = \"ali\"\n)\n\ntype testState struct {\n\t*State\n}\n\nfunc withState(t *testing.T, fn func(state *testState)) {\n\ttmpDir, err := ioutil.TempDir(\"\", \"brig-endpoints-test-userdb\")\n\trequire.Nil(t, err)\n\n\tdefer func() {\n\t\tos.RemoveAll(tmpDir)\n\t}()\n\n\tcfg, err := config.Open(nil, defaults.Defaults, config.StrictnessPanic)\n\trequire.Nil(t, err)\n\n\tfs, err := catfs.NewFilesystem(\n\t\tcatfs.NewMemFsBackend(),\n\t\tfilepath.Join(tmpDir, \"fs\"),\n\t\ttestGwUser,\n\t\tfalse,\n\t\tcfg.Section(\"fs\"),\n\t\tnil,\n\t\tnil,\n\t)\n\trequire.Nil(t, err)\n\n\tdbPath := filepath.Join(tmpDir, \"user\")\n\n\trapi := remotesapi.NewMock(\"ali\", \"alisfingerprint\")\n\tuserDb, err := db.NewUserDatabase(dbPath)\n\trequire.Nil(t, err)\n\n\tstate, err := NewState(\n\t\tfs,\n\t\trapi,\n\t\tcfg.Section(\"gateway\"),\n\t\tNewEventsHandler(rapi, nil),\n\t\tnil,\n\t\tuserDb,\n\t)\n\n\trequire.Nil(t, err)\n\n\tstate.UserDatabase().Add(\"ali\", \"ila\", nil, nil)\n\tfn(&testState{state})\n\n\trequire.NoError(t, state.Close())\n\trequire.NoError(t, state.fs.Close())\n\trequire.NoError(t, userDb.Close())\n}\n\nfunc mustEncodeBody(t *testing.T, v interface{}) io.Reader {\n\tbuf := &bytes.Buffer{}\n\trequire.Nil(t, json.NewEncoder(buf).Encode(v))\n\treturn buf\n}\n\nfunc mustDecodeBody(t *testing.T, body io.Reader, v interface{}) {\n\tdata, err := ioutil.ReadAll(body)\n\trequire.Nil(t, err)\n\trequire.Nil(t, json.NewDecoder(bytes.NewReader(data)).Decode(v))\n}\n\nfunc (s *testState) mustRun(t *testing.T, hdl http.Handler, verb, url string, jsonBody interface{}) *http.Response {\n\treq := httptest.NewRequest(verb, url, mustEncodeBody(t, jsonBody))\n\trsw := httptest.NewRecorder()\n\n\tuser, err := s.userDb.Get(\"ali\")\n\trequire.Nil(t, err)\n\n\treq = req.WithContext(context.WithValue(req.Context(), dbUserKey(\"brig.db_user\"), user))\n\tsetSession(s.store, \"ali\", rsw, req)\n\thdl.ServeHTTP(rsw, req)\n\treturn rsw.Result()\n}\n\nfunc (s *testState) mustChangeFolders(t *testing.T, folders ...string) {\n\trequire.Nil(t, s.userDb.Remove(\"ali\"))\n\trequire.Nil(t, s.userDb.Add(\"ali\", \"ila\", folders, nil))\n}\n" }, { "path": "gateway/endpoints/undelete.go", "content": "package endpoints\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\n\t\"github.com/sahib/brig/gateway/db\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// UndeleteHandler implements http.Handler.\ntype UndeleteHandler struct {\n\t*State\n}\n\n// NewUndeleteHandler creates a new undelete handler.\nfunc NewUndeleteHandler(s *State) *UndeleteHandler {\n\treturn &UndeleteHandler{State: s}\n}\n\n// UndeleteRequest is the request that can be sent to this endpoint as JSON.\ntype UndeleteRequest struct {\n\t// Path to create.\n\tPath string `json:\"path\"`\n}\n\nfunc (uh *UndeleteHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tif !checkRights(w, r, db.RightFsEdit) {\n\t\treturn\n\t}\n\n\tundelReq := UndeleteRequest{}\n\tif err := json.NewDecoder(r.Body).Decode(&undelReq); err != nil {\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"bad json\")\n\t\treturn\n\t}\n\n\tpath := prefixRoot(undelReq.Path)\n\tif !uh.validatePath(path, w, r) {\n\t\tjsonifyErrf(w, http.StatusUnauthorized, \"path forbidden\")\n\t\treturn\n\t}\n\n\tif err := uh.fs.Undelete(path); err != nil {\n\t\tlog.Debugf(\"failed to undelete %s: %v\", path, err)\n\t\tfmt.Println(err)\n\t\tjsonifyErrf(w, http.StatusInternalServerError, \"failed to undelete\")\n\t\treturn\n\t}\n\n\tmsg := fmt.Sprintf(\"undeleted »%s«\", path)\n\tif !uh.commitChange(msg, w, r) {\n\t\treturn\n\t}\n\n\tjsonifySuccess(w)\n}\n" }, { "path": "gateway/endpoints/undelete_test.go", "content": "package endpoints\n\nimport (\n\t\"net/http\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\ntype undeleteResponse struct {\n\tSuccess bool `json:\"success\"`\n\tMessage string `json:\"message\"`\n}\n\nfunc TestUndeleteEndpointSuccess(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\trequire.Nil(t, s.fs.Touch(\"/test\"))\n\t\trequire.Nil(t, s.fs.Touch(\"/dir\"))\n\t\trequire.Nil(t, s.fs.MakeCommit(\"create\"))\n\t\trequire.Nil(t, s.fs.Remove(\"/test\"))\n\t\trequire.Nil(t, s.fs.Remove(\"/dir\"))\n\t\trequire.Nil(t, s.fs.MakeCommit(\"remove\"))\n\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewUndeleteHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/undelete\",\n\t\t\t&UndeleteRequest{\n\t\t\t\tPath: \"/test\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\n\t\tundeleteResp := &undeleteResponse{}\n\t\tmustDecodeBody(t, resp.Body, &undeleteResp)\n\t\trequire.Equal(t, true, undeleteResp.Success)\n\n\t\tinfo, err := s.fs.Stat(\"/test\")\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, \"/test\", info.Path)\n\t\trequire.Equal(t, false, info.IsDir)\n\t})\n}\n\nfunc TestUndeleteEndpointInvalidPath(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\ts.mustChangeFolders(t, \"/something/else\")\n\t\tresp := s.mustRun(\n\t\t\tt,\n\t\t\tNewUndeleteHandler(s.State),\n\t\t\t\"POST\",\n\t\t\t\"http://localhost:5000/api/v0/undelete\",\n\t\t\t&UndeleteRequest{\n\t\t\t\tPath: \"/test\",\n\t\t\t},\n\t\t)\n\n\t\trequire.Equal(t, http.StatusUnauthorized, resp.StatusCode)\n\t})\n}\n" }, { "path": "gateway/endpoints/upload.go", "content": "package endpoints\n\nimport (\n\t\"fmt\"\n\t\"net/http\"\n\t\"path\"\n\n\t\"github.com/sahib/brig/gateway/db\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// UploadHandler implements http.Handler.\ntype UploadHandler struct {\n\t*State\n}\n\n// NewUploadHandler returns a new UploadHandler.\nfunc NewUploadHandler(s *State) *UploadHandler {\n\treturn &UploadHandler{State: s}\n}\n\nfunc (uh *UploadHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\tif !checkRights(w, r, db.RightFsEdit) {\n\t\tfmt.Println(\"BAD RIGHTS\")\n\t\treturn\n\t}\n\n\troot := r.URL.Query().Get(\"root\")\n\tif root == \"\" {\n\t\troot = \"/\"\n\t} else {\n\t\troot = prefixRoot(root)\n\t}\n\n\tif err := r.ParseMultipartForm(1 * 1024 * 1024); err != nil {\n\t\tlog.Debugf(\"upload: bad multipartform: %v\", err)\n\t\tjsonifyErrf(w, http.StatusBadRequest, \"failed to parse mutlipart form: %v\", err)\n\t\treturn\n\t}\n\n\t// Remove the cached files in /tmp\n\tdefer r.MultipartForm.RemoveAll()\n\n\tpaths := []string{}\n\n\tfor _, headers := range r.MultipartForm.File {\n\t\tfor _, header := range headers {\n\t\t\tpath := path.Join(root, header.Filename)\n\t\t\tfd, err := header.Open()\n\t\t\tif err != nil {\n\t\t\t\tlog.Debugf(\"upload: bad header: %v\", err)\n\t\t\t\tjsonifyErrf(w, http.StatusBadRequest, \"failed to open file: %v\", header.Filename)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tif !uh.validatePath(path, w, r) {\n\t\t\t\tjsonifyErrf(w, http.StatusUnauthorized, \"unauthorized\")\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tif err := uh.fs.Stage(path, fd); err != nil {\n\t\t\t\tlog.Debugf(\"upload: could not stage: %v\", err)\n\t\t\t\tjsonifyErrf(w, http.StatusBadRequest, \"failed to insert file: %v\", path)\n\t\t\t\tfd.Close()\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tpaths = append(paths, path)\n\t\t\tfd.Close()\n\t\t}\n\t}\n\n\tif len(paths) > 0 {\n\t\tmsg := fmt.Sprintf(\"uploaded »%s«\", paths[0])\n\t\tif len(paths) > 1 {\n\t\t\tmsg += fmt.Sprintf(\" and %d more\", len(paths)-1)\n\t\t}\n\n\t\tif !uh.commitChange(msg, w, r) {\n\t\t\treturn\n\t\t}\n\t}\n\n\tjsonifySuccess(w)\n}\n" }, { "path": "gateway/endpoints/upload_test.go", "content": "package endpoints\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"io/ioutil\"\n\t\"mime/multipart\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"net/url\"\n\t\"path\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc mustDoUpload(t *testing.T, s *testState, name string, data []byte) *http.Response {\n\tbody := &bytes.Buffer{}\n\twriter := multipart.NewWriter(body)\n\tpart, err := writer.CreateFormFile(\"file\", path.Base(name))\n\trequire.Nil(t, err)\n\n\t_, err = part.Write(data)\n\trequire.Nil(t, err)\n\trequire.Nil(t, writer.Close())\n\n\treq := httptest.NewRequest(\n\t\t\"POST\",\n\t\t\"/api/v0/upload?root=\"+url.QueryEscape(path.Dir(name)),\n\t\tbody,\n\t)\n\tuser, err := s.userDb.Get(\"ali\")\n\trequire.Nil(t, err)\n\treq = req.WithContext(context.WithValue(req.Context(), dbUserKey(\"brig.db_user\"), user))\n\n\treq.Header.Set(\"Content-Type\", writer.FormDataContentType())\n\trsw := httptest.NewRecorder()\n\tsetSession(s.store, \"ali\", rsw, req)\n\tNewUploadHandler(s.State).ServeHTTP(rsw, req)\n\treturn rsw.Result()\n}\n\nfunc TestUploadSuccess(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\trequire.Nil(t, s.fs.Mkdir(\"/sub\", true))\n\t\tresp := mustDoUpload(t, s, \"/sub/new_file.png\", []byte(\"hello\"))\n\n\t\trequire.Equal(t, http.StatusOK, resp.StatusCode)\n\t\tentries, err := s.fs.List(\"/sub\", 1)\n\t\trequire.Nil(t, err)\n\t\trequire.Len(t, entries, 1)\n\n\t\tstream, err := s.fs.Cat(\"/sub/new_file.png\")\n\t\trequire.Nil(t, err)\n\n\t\tdata, err := ioutil.ReadAll(stream)\n\t\trequire.Nil(t, err)\n\t\trequire.Equal(t, []byte(\"hello\"), data)\n\t})\n}\n\nfunc TestUploadForbidden(t *testing.T) {\n\twithState(t, func(s *testState) {\n\t\ts.mustChangeFolders(t, \"/public\")\n\t\tresp := mustDoUpload(t, s, \"/sub/new_file.png\", []byte(\"hello\"))\n\n\t\trequire.Equal(t, http.StatusUnauthorized, resp.StatusCode)\n\t})\n}\n" }, { "path": "gateway/endpoints/util.go", "content": "package endpoints\n\nimport (\n\t\"context\"\n\t\"encoding/base64\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"path\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/gorilla/securecookie\"\n\t\"github.com/gorilla/sessions\"\n\t\"github.com/sahib/brig/catfs\"\n\tie \"github.com/sahib/brig/catfs/errors\"\n\t\"github.com/sahib/brig/events\"\n\t\"github.com/sahib/brig/gateway/db\"\n\t\"github.com/sahib/brig/gateway/remotesapi\"\n\t\"github.com/sahib/config\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// State is a helper struct that contains all API objects that might be useful\n// to the endpoint implementation. It does not serve other purposes.\ntype State struct {\n\tfs *catfs.FS\n\trapi remotesapi.RemotesAPI\n\tcfg *config.Config\n\tev *events.Listener\n\tevHdl *EventsHandler\n\tstore *sessions.CookieStore\n\tuserDb *db.UserDatabase\n}\n\nfunc readOrInitKeyFromConfig(cfg *config.Config, keyName string, keyLen int) ([]byte, error) {\n\tkeyStr := cfg.String(keyName)\n\tif keyStr == \"\" {\n\t\tkeyData := securecookie.GenerateRandomKey(keyLen)\n\t\tcfg.SetString(keyName, base64.StdEncoding.EncodeToString(keyData))\n\t\treturn keyData, nil\n\t}\n\n\treturn base64.StdEncoding.DecodeString(keyStr)\n}\n\n// NewState creates a new state object.\n// events.Listener can be set later with SetEventListener.\nfunc NewState(\n\tfs *catfs.FS,\n\trapi remotesapi.RemotesAPI,\n\tcfg *config.Config,\n\tevHdl *EventsHandler,\n\tev *events.Listener,\n\tuserDb *db.UserDatabase,\n) (*State, error) {\n\tauthKey, err := readOrInitKeyFromConfig(cfg, \"auth.session-authentication-key\", 64)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tencKey, err := readOrInitKeyFromConfig(cfg, \"auth.session-encryption-key\", 32)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Generated here, but used by the server:\n\t_, err = readOrInitKeyFromConfig(cfg, \"auth.session-csrf-key\", 32)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &State{\n\t\tfs: fs,\n\t\trapi: rapi,\n\t\tcfg: cfg,\n\t\tevHdl: evHdl,\n\t\tstore: sessions.NewCookieStore(authKey, encKey),\n\t\tuserDb: userDb,\n\t}, nil\n}\n\n// Close cleans up any potentially open resource.\nfunc (s *State) Close() error {\n\ts.evHdl.Shutdown()\n\treturn nil\n}\n\n// UserDatabase returns the currently opened user database.\nfunc (s *State) UserDatabase() *db.UserDatabase {\n\treturn s.userDb\n}\n\nfunc (s *State) publishFsEvent(req *http.Request) {\n\tif s.evHdl != nil {\n\t\tctx, cancel := context.WithTimeout(req.Context(), 5*time.Second)\n\t\tdefer cancel()\n\n\t\ts.evHdl.Notify(ctx, \"fs\")\n\t}\n\n\tif s.ev == nil {\n\t\treturn\n\t}\n\n\tlog.Debugf(\"publishing fs event from gateway\")\n\tev := events.Event{\n\t\tType: events.FsEvent,\n\t}\n\n\tif err := s.ev.PublishEvent(ev); err != nil {\n\t\tlog.Warningf(\"failed to publish filesystem change event: %v\", err)\n\t}\n}\n\nfunc prefixRoot(nodePath string) string {\n\tif strings.HasPrefix(nodePath, \"/\") {\n\t\treturn nodePath\n\t}\n\n\treturn \"/\" + nodePath\n}\n\nfunc buildFolderCache(folders []string) map[string]bool {\n\tfolderCache := make(map[string]bool)\n\tfor _, folder := range folders {\n\t\tfolderCache[prefixRoot(path.Clean(folder))] = true\n\t}\n\n\treturn folderCache\n}\n\nfunc (s *State) pathIsVisible(nodePath string, w http.ResponseWriter, r *http.Request) bool {\n\tnodePath = prefixRoot(path.Clean(nodePath))\n\tif s.validatePath(nodePath, w, r) {\n\t\treturn true\n\t}\n\n\tname := getUserName(s.store, w, r)\n\tif name == \"\" {\n\t\treturn false\n\t}\n\n\tuser, err := s.userDb.Get(name)\n\tif err != nil {\n\t\treturn false\n\t}\n\n\tfolderCache := buildFolderCache(user.Folders)\n\tif err != nil {\n\t\tlog.Debugf(\"failed to build folder cache: %v\", err)\n\t\treturn false\n\t}\n\n\t// Go over all folders, and see if we have some allowed folder\n\t// that we need to display \"on the way\". This could be probably\n\t// made faster if we ever need to.\n\tfor folder, isValid := range folderCache {\n\t\tif !isValid {\n\t\t\tcontinue\n\t\t}\n\n\t\t// Example case:\n\t\t// folder = /nested/something\n\t\t// nodePath = /nested\n\t\t// (also handles if folder == nodePath)\n\t\t//\n\t\t// Other case (folder = /nested, nodePath = /nested/something)\n\t\t// is already handled by calling validatePath() above.\n\t\tif strings.HasPrefix(folder, nodePath) {\n\t\t\treturn true\n\t\t}\n\t}\n\n\t// There is no valid prefix at all.\n\treturn false\n}\n\nfunc (s *State) validatePath(nodePath string, w http.ResponseWriter, r *http.Request) bool {\n\tif !strings.HasPrefix(nodePath, \"/\") {\n\t\treturn false\n\t}\n\n\tname := getUserName(s.store, w, r)\n\tif name == \"\" {\n\t\treturn false\n\t}\n\n\tuser, err := s.userDb.Get(name)\n\tif err != nil {\n\t\treturn false\n\t}\n\n\t// At this point we know that the user is logged in.\n\treturn s.validatePathForUser(nodePath, user, w, r)\n}\n\nfunc (s *State) validatePathForUser(nodePath string, user db.User, w http.ResponseWriter, r *http.Request) bool {\n\tcurr := prefixRoot(nodePath)\n\tfolderCache := buildFolderCache(user.Folders)\n\n\tfor curr != \"\" {\n\t\tif folderCache[curr] {\n\t\t\treturn true\n\t\t}\n\n\t\tnext := path.Dir(curr)\n\t\tif curr == \"/\" && next == curr {\n\t\t\t// We've gone up too much:\n\t\t\tbreak\n\t\t}\n\n\t\tcurr = next\n\t}\n\n\t// No fitting path found:\n\treturn false\n}\n\n//////////////////////\n\nfunc jsonify(w http.ResponseWriter, statusCode int, data interface{}) {\n\tw.WriteHeader(statusCode)\n\tw.Header().Set(\"Content-Type\", \"application/json\")\n\n\tif err := json.NewEncoder(w).Encode(data); err != nil {\n\t\tlog.Warningf(\"failed to encode json: %v\", err)\n\t\tw.Write([]byte(\n\t\t\t\"{\\\"success\\\": false, \\\"message\\\": \\\"failed to encode json response\\\"}\",\n\t\t))\n\t\tw.WriteHeader(500)\n\t\treturn\n\t}\n}\n\nfunc jsonifyErrf(w http.ResponseWriter, statusCode int, format string, data ...interface{}) {\n\tmsg := fmt.Sprintf(format, data...)\n\tsuccess := false\n\tif statusCode >= 200 && statusCode < 400 {\n\t\tsuccess = true\n\t} else {\n\t\t// TODO: also pass request and output the url in this log message.\n\t\tlog.Debugf(\"failed to respond: %v\", msg)\n\t}\n\n\tjsonify(w, statusCode, struct {\n\t\tSuccess bool `json:\"success\"`\n\t\tMessage string `json:\"message\"`\n\t}{\n\t\tSuccess: success,\n\t\tMessage: msg,\n\t})\n}\n\nfunc jsonifySuccess(w http.ResponseWriter) {\n\tjsonifyErrf(w, http.StatusOK, \"success\")\n}\n\nfunc (s *State) commitChange(msg string, w http.ResponseWriter, r *http.Request) bool {\n\tname := getUserName(s.store, w, r)\n\tfullMsg := fmt.Sprintf(\"gateway: »%s« %s\", name, msg)\n\n\tif err := s.fs.MakeCommit(fullMsg); err != nil {\n\t\tif err != ie.ErrNoChange {\n\t\t\tlog.Warningf(\"could not commit: %v\", err)\n\t\t\tjsonifyErrf(w, http.StatusInternalServerError, \"could not commit\")\n\t\t\treturn false\n\t\t}\n\n\t\t// There was no change. No need to notify.\n\t\treturn true\n\t}\n\n\ts.evHdl.Notify(r.Context(), \"fs\")\n\treturn true\n}\n\n///////\n\ntype secureMiddleware struct {\n\t*State\n\tSubHandler http.Handler\n}\n\nfunc (sm *secureMiddleware) ServeHTTP(w http.ResponseWriter, r *http.Request) {\n\thdr := w.Header()\n\n\t// Do not let browsers guess the content type:\n\t// https://en.wikipedia.org/wiki/Content_sniffing\n\thdr.Set(\"X-Content-Type-Options\", \"nosniff\")\n\n\t// https://security.stackexchange.com/questions/121796/what-security-implications-does-dns-prefetching-have\n\thdr.Set(\"X-DNS-Prefetch-Control\", \"off\")\n\n\t// Do not allow