Repository: dat-ecosystem/dat Branch: master Commit: 14885d660adc Files: 72 Total size: 159.0 KB Directory structure: gitextract_aa8sr91t/ ├── .github/ │ ├── FUNDING.yml │ └── issue_template.md ├── .gitignore ├── .travis.yml ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── appveyor.yml ├── bin/ │ └── cli.js ├── changelog.md ├── download.sh ├── index.js ├── package.json ├── package.sh ├── scripts/ │ └── auth-server.js ├── snap/ │ └── snapcraft.yaml ├── src/ │ ├── commands/ │ │ ├── auth/ │ │ │ ├── login.js │ │ │ ├── logout.js │ │ │ ├── register.js │ │ │ └── whoami.js │ │ ├── clone.js │ │ ├── create.js │ │ ├── doctor.js │ │ ├── keys.js │ │ ├── log.js │ │ ├── publish.js │ │ ├── pull.js │ │ ├── status.js │ │ ├── sync.js │ │ └── unpublish.js │ ├── extensions.js │ ├── lib/ │ │ ├── archive.js │ │ ├── discovery-exit.js │ │ ├── download.js │ │ ├── exit.js │ │ ├── import-progress.js │ │ ├── network.js │ │ ├── selective-sync.js │ │ ├── serve-http.js │ │ └── stats.js │ ├── parse-args.js │ ├── registry.js │ ├── ui/ │ │ ├── archive.js │ │ ├── components/ │ │ │ ├── download.js │ │ │ ├── import-progress.js │ │ │ ├── network.js │ │ │ ├── sources.js │ │ │ └── warnings.js │ │ ├── create.js │ │ ├── elements/ │ │ │ ├── key.js │ │ │ ├── pluralize.js │ │ │ └── version.js │ │ └── status.js │ └── usage.js └── test/ ├── auth.js ├── clone.js ├── create.js ├── dat-node.js ├── doctor.js ├── fixtures/ │ ├── all_hour.csv │ └── folder/ │ └── nested/ │ └── hello.txt ├── helpers/ │ ├── auth-server.js │ ├── index.js │ └── spawn.js ├── http.js ├── keys.js ├── pull.js ├── share.js ├── sync-owner.js ├── sync-remote.js └── usage.js ================================================ FILE CONTENTS ================================================ ================================================ FILE: .github/FUNDING.yml ================================================ open_collective: dat ================================================ FILE: .github/issue_template.md ================================================ I am reporting: # Bug Report Please give us details about your installation to assist you. Run `dat -v` to see the version of Dat you are using. * Operating system: * Node Version: * Dat Version: ### Expected behavior ### Actual behavior ### Debug Logs ``` ``` ================================================ FILE: .gitignore ================================================ node_modules .DS_Store tmp .idea data yarn.lock test/fixtures/.dat test/fixtures/dat.json test/**.db test/.datrc-test dist builds ================================================ FILE: .travis.yml ================================================ language: node_js node_js: - 'lts/*' - '12' - 'node' sudo: false script: - npm test notifications: irc: channels: - chat.freenode.net#datbots template: - '%{repository_slug} - %{commit_subject} - %{result} - %{build_url}' skip_join: true on_success: change on_failure: always before_deploy: npm run package deploy: provider: releases api_key: secure: GF+Ehh9kDu2m+KqSzciZRQmUfubnVGDEfxZKVX+psesKoxxDSq8/wkl7g1yR2H8DO0dg3lW8opbsKbfOOUWztyIfFxFukgwKIawUd7Krtr4XQLyywq49NdYARKP6bSxeEb8N3xVTo5fuq104KT0mMUB9di/iunsO/ITOzbCZyWE= skip_cleanup: true file_glob: true file: dist/* on: repo: datproject/dat node: '12' tags: true ================================================ FILE: CODE_OF_CONDUCT.md ================================================ Our code of conduct is under review in [this repo](https://github.com/datproject/Code-of-Conduct) - please check it out and give us feedback! ================================================ FILE: CONTRIBUTING.md ================================================ # Welcome to Dat! Please take a second to read over this before opening an issue. Providing complete information upfront will help us address any issue (and ship new features!) faster. Our time is limited, please respect it and create complete issues. We are available to chat in IRC at #dat. You can join [via Gitter](https://gitter.im/datproject/discussions). You can also [view and search](https://botbot.me/freenode/dat/) chat logs. We have a [faq](https://docs.datproject.org/faq) section on our docs that may address non-bug questions. ## Opening an Issue Please read this section before opening a new issue. `dat` is composed of many modules and this repository is just one of them. If you know your issue is with another module, we prefer you open it in that repository. There may be an exiting issue in another repository. If you aren't sure, then this is the perfect place. Any new issues should be *actionable*. If your issue cannot be solved (and thus closed) please reconsider opening it in our discussion repository or rewording it. ### Bug Reports A perfect bug report would have the following: 1. Summary of the issue you are experiencing. 2. Details on what versions of node and dat you have (`node -v` and `dat -v`). 3. A simple repeatable test case for us to run. Please try to run through it 2-3 times to ensure it is completely repeatable. We would like to avoid issues that require a follow up questions to identify the bug. These follow ups are difficult to do unless we have a repeatable test case. We can't all be perfect =). Do as much as you can and we'll try to help you with the rest. If we are slow to respond, a more complete bug report will help. ### Feature Requests Feature requests are more than welcome. Please search exiting issues (open and closed) to make sure it is not a duplicate. A good feature request would have examples of how to use it, some detailed use cases, and any concerns or possible edge cases. Keep in mind we have specific use cases we are building for (namely scientific data sharing). If, your feature request does not fit within that use case it may not be prioritized or implemented. ### General Discussion Issues We prefer to be able to close issues in this repository, which does not lend itself to discussion type questions. Open discussion type issue in the [datproject/discussions](https://github.com/datproject/discussions/issues) repository. ## For Developers Please read these guidelines if you are interested in contributing to Dat. ### Submitting pull requests Before taking the time to code something, feel free to open an issue first proposing your idea to other contributors, that way you can get feedback on the idea before taking time to write precious code. For any new functionality we like to see: * unit tests so that we can improve long term maintenance and catch regressions in the future * updates to the [change log](http://keepachangelog.com/) and relevant documentation ### For Collaborators Make sure to get a `:thumbsup:`, `+1` or `LGTM` from another collaborator before merging a PR. If you aren't sure if a release should happen, open an issue. Release process: - make sure the tests pass - Update changelog - `npm version ` - `git push && git push --tags` - `npm publish` ### Development workflow We use and write a lot of node modules and it introduces a bit of a learning curve when working on multiple modules simultaneously. There are lots of different and valid solutions to working on lots of modules at once, this is just one way. #### Developing inside a node_modules folder First make sure you are comfortable with [how require works](https://github.com/maxogden/art-of-node#how-require-works) in node. We recommend creating a folder somewhere manually called `node_modules`. For example in `~/code/node_modules`. Clone all of your git copies of modules that you want to work on into here, so for example: - `~/code/node_modules/dat` - `~/code/node_modules/hyperdrive` When you run `npm install` inside of `~/code/node_modules/dat`, dat will get its own copy of `hyperdrive` (one if its dependencies) inside `~/code/node_modules/dat/node_modules`. However, if you encounter a bug in hyperdrive that you need to fix, but you want to test your fix in dat, you want dat to use your git copy of hyperdrive at `~/code/node_modules/hyperdrive` and not the npm copy of hyperdrive at `~/code/node_modules/dat/node_modules/hyperdrive`. How do you get dat to use the git copy of hyperdrive? Just delete the npm copy! ``` rm -rf ~/code/node_modules/dat/node_modules/hyperdrive ``` Now when you run dat, and it tries to `require('hyperdrive')` it first looks in its own `node_modules` folder at `~/code/node_modules/dat/node_modules` but doesnt find hyperdrive. So it goes up to `~/code/node_modules` and finds `hyperdrive` there and uses that one, your git copy. If you want to switch back to an npm copy, just run `npm install` inside `~/code/node_modules/dat/` and npm will download any missing modules into `~/code/node_modules/dat/node_modules` but wont touch anything in `~/code/node_modules`. This might seem a bit complicated at first, but is simple once you get the hang of it. Here are some rules to help you get started: - Never make any meaningful edits to code inside an "npm-managed" node_modules folder (such as `~/code/node_modules/dat/node_modules`), because when you run `npm install` inside those folders it could inadvertently delete all of your edits when installing an updated copy of a module. This has happened to me many times, so I just always use my git copy and delete the npm copy (as described above) to make edits to a module. - You should never need to run any npm commands in terminal when at your "manually managed"" node_modules folder at `~/code/node_modules`. Never running npm commands at that folder also prevents npm from accidentally erasing your git copies of modules - The location of your "manually managed" node_modules folder should be somewhere isolated from your normal require path. E.g. if you put it at `~/node_modules`, then when you run `npm install dat` at `~/Desktop` npm might decide to erase your git copy of dat at `~/node_modules/dat` and replace it with a copy from npm, which could make you lose work. Putting your manually managed `node_modules` folder in a sub-folder like `~/code` gets it "out of the way" and prevents accidents like that from happening. ================================================ FILE: LICENSE ================================================ Copyright (c) 2015 Max Ogden. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ================================================ FILE: README.md ================================================ More info on active projects and modules at [dat-ecosystem.org](https://dat-ecosystem.org/) --- # Dat > npm install -g dat Use `dat` command line to share files with version control, back up data to servers, browse remote files on demand, and automate long-term data preservation. `dat` is the first application based upon the [Hypercore Protocol](https://github.com/hypercore-protocol), and drove the architectural design through iterative development between 2014 and 2017. There exists a large community around it. [][Dat Project] Have questions? Join our chat via IRC or Gitter: [![#dat IRC channel on freenode][irc-badge]][irc-channel] [![datproject/discussions][gitter-badge]][gitter-chat] ### Thanks to our financial supporters! ![OpenCollective](https://opencollective.com/dat/tiers/maintainer.svg?avatarHeight=36&width=600") ### Table of Contents - [Installation](#installation) - [Getting Started](#getting-started) - [Using Dat](#usage) - [Troubleshooting](#troubleshooting) - [Javascript API](#js-api) - [For Developers](#for-developers) ## Installation Dat can be used as a command line tool or a javascript library: * Install the `$ dat` CLI to use in the command line. * [require('dat')][dat-node] - dat-node, a library for downloading and sharing dat archives in javascript apps. ### Installing the `$ dat` command line tool The recommended way to install dat is through a single file binary distribution version of `dat` with the one line install command below. The binary includes a copy of node and dat packaged inside a single file, so you just have to download one file in order to start sharing data, with no other dependencies needed on your system: ``` wget -qO- https://raw.githubusercontent.com/datproject/dat/master/download.sh | bash ``` ## Next version Try the next version of dat! This version (14.0.0) is not compatible with older versions (13.x) and below, and works on node v12. ``` npm install -g dat@next ``` Maintainers wanted! #### NPM Prerequisites * **Node**: You'll need to [install Node JS][install-node] before installing Dat. Dat needs `node` version 4 or above and `npm` installed. You can run `node -v` to check your version. * **npm**: `npm` is installed with node. You can run `npm -v` to make sure it is installed. Once you have `npm` ready, install `dat` from npm with the `--global, -g` option, `npm install -g dat`. ## Getting started #### What is Dat? Share, backup, and publish your filesystem. You can turn any folder on your computer into a dat. Dat scans your folder, allowing you to: * Track your files with automatic version history. * Share files with others over a secure peer to peer network. * Automate live backups to external HDs or remote servers. * Publish and share files with built in HTTP server. Dat allows you to focus on the fun work without worrying about moving files around. **Secure**, **distributed**, **fast**. * Documentation: [docs.datproject.org](https://docs.datproject.org) * [Dat white paper](https://github.com/datprotocol/whitepaper/blob/master/dat-paper.pdf) ##### Desktop Applications Rather not use the command line? Check out these options: * [Beaker Browser] - An experimental p2p browser with built-in support for the Hypercore Protocol. * [Dat Desktop](https://github.com/datproject/dat-desktop) - A desktop app to manage multiple dats on your desktop machine. ### JS Library Add Dat to your `package.json`, `npm install dat --save`. Dat exports the [dat-node] API via `require('dat')`. Use it in your javascript applications! Dat Desktop and Dat command line both use dat-node to share and download dats. Full API documentation is available in the [dat-node] repository on Github. We have Dat installed, let's use it! Dat's unique design works wherever you store your data. You can create a new dat from any folder on your computer. A dat is some files from your computer and a `.dat` folder. Each dat has a unique `dat://` link. With your dat link, other users can download your files and live sync any updates. ### Sharing Data You can start sharing your files with a single command. Unlike `git`, you do not have to initialize a repository first, `dat share` or simply `dat` will do that for you: ``` dat ``` Use `dat` to create a dat and sync your files from your computer to other users. Dat scans your files inside ``, creating metadata in `/.dat`. Dat stores the public link, version history, and file information inside the dat folder. `dat sync` and `dat share` are aliases for the same command. ![share-gif] ### Downloading Data ``` dat dat:// ``` Use `dat` to download files from a remote computer sharing files with Dat. This will download the files from `dat://` to your ``. The download exits after it completes but you can continue to update the files later after the clone is done. Use `dat pull` to update new files or `dat sync` to live sync changes. `dat clone` is an alias for the same command. ![clone-gif] ### Misc Commands A few other highlights. Run `dat help` to see the full usage guide. * `dat create` or `dat init` - Create an empty dat and `dat.json` file. * `dat log ~/data/dat-folder/` or `dat log dat://` - view the history and metadata information for a dat. ### Quick Demos To get started using Dat, you can try downloading a dat and then sharing a dat of your own. #### Download Demo We made a demo folder just for this exercise. Inside the demo folder is a `dat.json` file and a gif. We shared these files via Dat and now you can download them with our dat key! Similar to git, you can download somebody's dat by running `dat clone `. You can also specify the directory: ``` ❯ dat clone dat://778f8d955175c92e4ced5e4f5563f69bfec0c86cc6f670352c457943666fe639 ~/Downloads/dat-demo dat v13.5.0 Created new dat in /Users/joe/Downloads/dat-demo/.dat Cloning: 2 files (1.4 MB) 2 connections | Download 614 KB/s Upload 0 B/s dat sync complete. Version 4 ``` This will download our demo files to the `~/Downloads/dat-demo` folder. These files are being shared by a server over Dat (to ensure high availability) but you may connect to any number of users also hosting the content. You can also also view the files online: [datbase.org/778f8d955175c92e4ced5e4f5563f69bfec0c86cc6f670352c457943666fe639](https://datbase.org/778f8d955175c92e4ced5e4f5563f69bfec0c86cc6f670352c457943666fe639/). datbase.org can download files over Dat and display them on HTTP as long as someone is hosting it. The website temporarily caches data for any visited links (do not view your dat on datbase.org if you do not want us to cache your data). #### Sharing Demo Dat can share files from your computer to anywhere. If you have a friend going through this demo with you, try sharing to them! If not we'll see what we can do. Find a folder on your computer to share. Inside the folder can be anything, Dat can handle all sorts of files (Dat works with really big folders too!). First, you can create a new dat inside that folder. Using the `dat create` command also walks us through making a `dat.json` file: ``` ❯ dat create Welcome to dat program! You can turn any folder on your computer into a Dat. A dat is a folder with some magic. ``` This will create a new (empty) dat. Dat will print a link, share this link to give others access to view your files. Once we have our dat, run `dat ` to scan your files and sync them to the network. Share the link with your friend to instantly start downloading files. #### Bonus HTTP Demo Dat makes it really easy to share live files on a HTTP server. This is a cool demo because we can also see how version history works! Serve dat files on HTTP with the `--http` option. For example, `dat --http`, serves your files to a HTTP website with live reloading and version history! This even works for dats you're downloading (add the `--sparse` option to only download files you select via HTTP). The default HTTP port is 8080. *Hint: Use `localhost:8080/?version=10` to view a specific version.* Get started using Dat today with the `share` and `clone` commands or read below for more details. ## Usage The first time you run a command, a `.dat` folder is created to store the dat metadata. Once a dat is created, you can run all the commands inside that folder, similar to git. Dat keeps secret keys in the `~/.dat/secret_keys` folder. These are required to write to any dats you create. #### Creating a dat & dat.json ``` dat create [] ``` The create command prompts you to make a `dat.json` file and creates a new dat. Import the files with sync or share. Optionally bypass Title and Description prompt: ```sh dat create --title "MY BITS" --description "are ready to synchronize! 😎" ``` Optionally bypass `dat.json` creation: ```sh dat create --yes dat create -y ``` ### Sharing The quickest way to get started sharing files is to `share`: ``` ❯ dat dat://3e830227b4b2be197679ff1b573cc85e689f202c0884eb8bdb0e1fcecbd93119 Sharing dat: 24 files (383 MB) 0 connections | Download 0 B/s Upload 0 B/s Importing 528 files to Archive (165 MB/s) [=-----------------------------------------] 3% ADD: data/expn_cd.csv (403 MB / 920 MB) ``` ``` dat [] [--no-import] [--no-watch] ``` Start sharing your dat archive over the network. It will import new or updated files since you last ran `create` or `sync`. Dat watches files for changes and imports updated files. * Use `--no-import` to not import any new or updated files. * Use `--no-watch` to not watch directory for changes. `--import` must be true for `--watch` to work. #### Ignoring Files By default, Dat will ignore any files in a `.datignore` file, similar to git. Each file should be separated by a newline. Dat also ignores all hidden folders and files. Supports pattern wildcards (`/*.png`) and directory-wildcards (`/**/cache`). #### Selecting Files By default, Dat will download all files. If you want to only download a subset, you can create a `.datdownload` file which downloads only the files and folders specified. Each should be separated by a newline. ### Downloading Start downloading by running the `clone` command. This creates a folder, downloads the content and metadata, and a `.dat` folder inside. Once you started the download, you can resume at any time. ``` dat [] [--temp] ``` Clone a remote dat archive to a local folder. This will create a folder with the key name if no folder is specified. #### Downloading via `dat.json` key You can use a `dat.json` file to clone also. This is useful when combining Dat and git, for example. To clone a dat you can specify the path to a folder containing a `dat.json`: ``` git git@github.com:joehand/dat-clone-sparse-test.git dat ./dat-clone-sparse-test ``` This will download the dat specified in the `dat.json` file. #### Updating Downloaded Archives Once a dat is clone, you can run either `dat pull` or `dat sync` in the folder to update the archive. ``` dat pull [] ``` Download latest files and keep connection open to continue updating as remote source is updated. ### Shortcut commands * `dat ` will run `dat clone` for new dats or resume the existing dat in `` * `dat ` is the same as running `dat sync ` ### Key Management & Moving dats `dat keys` provides a few commands to help you move or backup your dats. Writing to a dat requires the secret key, stored in the `~/.dat` folder. You can export and import these keys between dats. First, clone your dat to the new location: * (original) `dat share` * (duplicate) `dat clone ` Then transfer the secret key: * (original) `dat keys export` - copy the secret key printed out. * (duplicate) `dat keys import` - this will prompt you for the secret key, paste it in here. ## Troubleshooting We've provided some troubleshooting tips based on issues users have seen. Please [open an issue][new-issue] or ask us in our [chat room][gitter-chat] if you need help troubleshooting and it is not covered here. If you have trouble sharing/downloading in a directory with a `.dat` folder, try deleting it and running the command again. #### Check Your Dat Version Knowing the version is really helpful if you run into any bugs, and will help us troubleshoot your issue. Check your Dat version: ``` dat -v ``` You should see the Dat semantic version printed, e.g. `14.0.0`. ### Installation Issues #### Node & npm To use the Dat command line tool you will need to have [node and npm installed][install-node-npm]. Make sure those are installed correctly before installing Dat. You can check the version of each: ``` node -v npm -v ``` #### Global Install The `-g` option installs Dat globally, allowing you to run it as a command. Make sure you installed with that option. * If you receive an `EACCES` error, read [this guide][fixing-npm-permissions] on fixing npm permissions. * If you receive an `EACCES` error, you may also install Dat with sudo: `sudo npm install -g dat`. * Have other installation issues? Let us know, you can [open an issue][new-issue] or ask us in our [chat room][gitter-chat]. ### Debugging Output If you are having trouble with a specific command, run with the debug environment variable set to `dat` (and optionally also `dat-node`). This will help us debug any issues: ``` DEBUG=dat,dat-node dat dat:// dir ``` ### Networking Issues Networking capabilities vary widely with each computer, network, and configuration. Whenever you run Dat there are several steps to share or download files with peers: 1. Discovering Peers 2. Connecting to Peers 3. Sending & Receiving Data With successful use, Dat will show `Connected to 1 peer` after connection. If you never see a peer connected, your network may be restricting discovery or connection. ## JS API You can use Dat in your javascript application: ```js var Dat = require('dat') Dat('/data', function (err, dat) { // use dat }) ``` **[Read more][dat-node] about the JS usage provided via `dat-node`.** ## For Developers Please see [guidelines on contributing] before submitting an issue or PR. This command line library uses [dat-node] to create and manage the archives and networking. If you'd like to build your own Dat application that is compatible with this command line tool, we suggest using dat-node. ### Installing from source Clone this repository and in a terminal inside of the folder you cloned run this command: ``` npm link ``` This should add a `dat` command line command to your PATH. Now you can run the `dat` command to try it out. The contribution guide also has more tips on our [development workflow]. * `npm run test` to run tests * `npm run auth-server` to run a local auth server for testing ## License BSD-3-Clause [Dat Project]: https://datproject.org [Code for Science & Society]: https://codeforscience.org [Dat white paper]: https://github.com/datproject/docs/blob/master/papers/dat-paper.pdf [Dat Desktop]: https://docs.datproject.org/install#desktop-application [Beaker Browser]: https://beakerbrowser.com [registry server]: https://github.com/datproject/datbase [share-gif]: https://raw.githubusercontent.com/datproject/docs/master/docs/assets/cli-share.gif [clone-gif]: https://raw.githubusercontent.com/datproject/docs/master/docs/assets/cli-clone.gif [Knight Foundation grant]: https://blog.datproject.org/2016/02/01/announcing-publicbits-org/ [dat-node]: https://github.com/datproject/dat-node [dat-ignore]: https://github.com/joehand/dat-ignore [new-issue]: https://github.com/datproject/dat/issues/new [dat#503]: https://github.com/datproject/dat/issues/503 [install-node]: https://nodejs.org/en/download/ [install-node-npm]: https://docs.npmjs.com/getting-started/installing-node [fixing-npm-permissions]: https://docs.npmjs.com/getting-started/fixing-npm-permissions [guidelines on contributing]: https://github.com/datproject/dat/blob/master/CONTRIBUTING.md [development workflow]: https://github.com/datproject/dat/blob/master/CONTRIBUTING.md#development-workflow [travis-badge]: https://travis-ci.org/datproject/dat.svg?branch=master [travis-build]: https://travis-ci.org/datproject/dat [appveyor-badge]: https://ci.appveyor.com/api/projects/status/github/datproject/dat?branch=master&svg=true [appveyor-build]: https://ci.appveyor.com/project/joehand/dat/branch/master [npm-badge]: https://img.shields.io/npm/v/dat.svg [npm-package]: https://npmjs.org/package/dat [irc-badge]: https://img.shields.io/badge/irc%20channel-%23dat%20on%20freenode-blue.svg [irc-channel]: https://webchat.freenode.net/?channels=dat [gitter-badge]: https://badges.gitter.im/Join%20Chat.svg [gitter-chat]: https://gitter.im/datproject/discussions ================================================ FILE: appveyor.yml ================================================ # Test against this version of Node.js environment: matrix: - nodejs_version: "6" - nodejs_version: "8" - nodejs_version: "10" # Install scripts. (runs after repo cloning) install: - ps: Install-Product node $env:nodejs_version - npm install test_script: # Output useful info for debugging. - node --version - npm --version - npm test build: off ================================================ FILE: bin/cli.js ================================================ #!/usr/bin/env node var subcommand = require('subcommand') var debug = require('debug')('dat') var usage = require('../src/usage') var pkg = require('../package.json') process.title = 'dat' // Check node version to make sure we support var NODE_VERSION_SUPPORTED = 4 var nodeMajorVer = process.version.match(/^v([0-9]+)\./)[1] var invalidNode = nodeMajorVer < NODE_VERSION_SUPPORTED if (invalidNode) exitInvalidNode() else { var notifier = require('update-notifier') notifier({ pkg: pkg }) .notify({ defer: true, isGlobal: true, boxenOpts: { align: 'left', borderColor: 'green', borderStyle: 'classic', padding: 1, margin: { top: 1, bottom: 1 } } }) } if (debug.enabled) { debug('Dat DEBUG mode engaged, enabling quiet mode') } var config = { defaults: [ { name: 'dir', abbr: 'd', help: 'set the directory for Dat' }, { name: 'logspeed', default: 400 }, { name: 'port', help: 'port to use for connections (default port: 3282 or first available)' }, { name: 'utp', default: true, boolean: true, help: 'use utp for discovery' }, { name: 'http', help: 'serve dat over http (default port: 8080)' }, { name: 'debug', default: !!process.env.DEBUG && !debug.enabled, boolean: true }, { name: 'quiet', default: debug.enabled, boolean: true }, // use quiet for dat debugging { name: 'sparse', default: false, boolean: true, help: 'download only requested data' }, { name: 'up', help: 'throttle upload bandwidth (1024, 1kb, 2mb, etc.)' }, { name: 'down', help: 'throttle download bandwidth (1024, 1kb, 2mb, etc.)' } ], root: { options: [ { name: 'version', boolean: true, default: false, abbr: 'v' } ], command: usage }, none: syncShorthand, commands: [ require('../src/commands/clone'), require('../src/commands/create'), require('../src/commands/log'), require('../src/commands/keys'), require('../src/commands/publish'), require('../src/commands/pull'), require('../src/commands/status'), require('../src/commands/sync'), require('../src/commands/unpublish'), require('../src/commands/auth/register'), require('../src/commands/auth/whoami'), require('../src/commands/auth/logout'), require('../src/commands/auth/login') ], usage: { command: showUsageOrRunExtension, option: { name: 'help', abbr: 'h' } }, aliases: { 'init': 'create', 'share': 'sync' }, // whitelist extensions for now extensions: [ 'store' ] } if (debug.enabled) { debug('dat', pkg.version) debug('node', process.version) } // Match Args + Run command var match = subcommand(config) match(alias(process.argv.slice(2))) function alias (argv) { var cmd = argv[0] if (!config.aliases[cmd]) return argv argv[0] = config.aliases[cmd] return argv } // CLI Shortcuts // Commands: // dat [] - clone/sync a key // dat - create dat + share a directory // dat function syncShorthand (opts) { if (!opts._.length) return usage(opts) debug('Sync shortcut command') debug('Trying extension', opts._[0]) // First try extension if (config.extensions.indexOf(opts._[0]) > -1) return require('../src/extensions')(opts) var parsed = require('../src/parse-args')(opts) // Download Key if (parsed.key) { // dat [] - clone/resume in [dir] debug('Clone sync') opts.dir = parsed.dir || parsed.key // put in `process.cwd()/key` if no dir opts.exit = opts.exit || false return require('../src/commands/clone').command(opts) } // Sync dir // dat - sync existing dat in {dir} if (parsed.dir) { opts.shortcut = true debug('Share sync') // Set default opts. TODO: use default opts in share opts.watch = opts.watch || true opts.import = opts.import || true return require('../src/commands/sync').command(opts) } // All else fails, show usage return usage(opts) } // This was needed so that we can show help messages from extensions function showUsageOrRunExtension (opts, help, usageMessage) { if (config.extensions.indexOf(opts._[0]) > -1) return require('../src/extensions')(opts) usage(opts, help, usageMessage) } function exitInvalidNode () { console.error('Node Version:', process.version) console.error('Unfortunately, we only support Node >= v4. Please upgrade to use Dat.') console.error('You can find the latest version at https://nodejs.org/') process.exit(0) } ================================================ FILE: changelog.md ================================================ # Change Log All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](https://semver.org/). ## [Unreleased] ## 13.13.0 * `dat pull --exit=NN` exits after `NN` number of seconds, when there are no updates to sync. ## 13.9.0 - 2017-10-11 ### Changed * Use [datbase.org](https://datbase.org) as default registry (instead of datproject.org) ## 13.8.2 - 2017-09-28 ### Fixed * Error not being handled (https://github.com/datproject/dat/issues/838) * Set `opts.debug` properly when using `DEBUG` that isn't `dat`. * Move discovery key to option in `dat keys` (#869) ## 13.8.1 - 2017-08-04 ### Fixes * Error not being handled (https://github.com/datproject/dat/issues/838) ## 13.8.0 - 2017-08-04 With this release, we are adding an exciting feature that really showcases how powerful Dat is, selective sync. Using the CLI you can now specify which files you want to download either with an option or using the `.datdownload` file. `dat sync` will download and keep updated on the selected files. This means you can put large datasets into Dat but have control over what files you download where. [Full release notes](https://github.com/datproject/dat/releases/tag/v13.8.0) ## Added * Selective Sync (https://github.com/datproject/dat/pull/834) * Key management (https://github.com/datproject/dat/pull/828) ## Changed * Commands run faster via lazy required modules (https://github.com/datproject/dat/pull/821) ## 13.7.0 - 2017-06-28 ## Added * Throttling - sometimes Dat goes too fast, so you can limit the upload + download speeds. (https://github.com/datproject/dat/pull/806) * Publish metadata to registry when publishing (https://github.com/datproject/dat/pull/812) ## Changed * Use dat-node http support directly (https://github.com/datproject/dat/pull/817) ## Fixed * Use npm package for registry testing. ## 13.6.0 - 2017-06-05 Full support for Dat registries! See our [full release notes](https://github.com/datproject/dat/releases/tag/v13.6.0). ### Added * Improved support for public Dat registries (https://github.com/datproject/dat/pull/794) * Add unpublish command ## 13.5.1 - 2017-05-30 ### Changed * Big documentation update! * Force bump dat-node for a deletion bug that was a bit overzealous in deleting files (https://github.com/mafintosh/hyperdrive/issues/167). ## 13.5.0 - 2017-05-25 ### Added * Dat version number is printed in header (https://github.com/datproject/dat/pull/788) * Add prompt and introduction to `dat create` command (https://github.com/datproject/dat/pull/782) and create dat.json file (https://github.com/datproject/dat/pull/765). * Tell user if new `.dat` was initialized. * Add `dat log` command to print archive history and size information (https://github.com/datproject/dat/pull/781). * Use `require('dat')` to get `dat-node` JS API (https://github.com/datproject/dat/pull/778). ### Changed * Default to upload true for `dat clone` and `dat pull`, enables better hole-punching (https://github.com/datproject/dat/pull/787). ### Fixed * Make argument parsing more consistent across commands (https://github.com/datproject/dat/pull/789) * Fix usage and help text (various). ## 13.4.1 - 2017-05-16 ### Added * Document sparse option in cli help * add node/dat version to debug ### Changed * Use share for shortcut (create new dat if not created) ### Fixed * use exit option on clone shortcut if specified * [various ui fixes](https://github.com/datproject/dat/pull/764) ## 13.4.0 - 2017-05-11 ### Added * Serve dat over http with `--http` option ## 13.3.0 - 2017-05-10 ### Added * Add `--sources` option for debugging network issues ## 13.2.0 - 2017-05-10 ### Added * Dat-* extensions ([#740](https://github.com/datproject/dat/pull/740)) * Ignore directories in import (dat-node v3.3.0) ## 13.1.1 - 2017-05-10 ### Fixed * Set directory for publish command ### Changed * Improve `--show-key` help output * Always show download progress bar and make language more clear. ## 13.1.0 - 2017-05-09 ### Fixed * Cleanup dat shortcut + directory creation * Check for any swarm.connecting before doing discovery failure. ### Added * Check node version, fail for anything older than node v4 (#669) * Add show-key option to display key on downloading cmds * `dat status` command to show key, file count, dir size, and archive version ## 13.0.0 - 2017-05-08 ### Changed * Upgrade to Hyperdrive v8/9 (SLEEP archive format) and Dat-node v2/3. See [dat-node release docs](https://github.com/datproject/dat-node/releases/tag/v2.0.0) for more info. * UI updates ## 12.0.3 - 2017-03-29 ### Fixed * Content progress for archives with history * Change `process.title` to `dat` from `dat-next` ### Changed * Use two decimals for content progress ## 12.0.2 - 2017-02-08 ### Fixed * Remove `hyperdrive-import-files` from dependencies (it is a dependency of `dat-node`). It was accidentally added. * Always verify on read to avoid replication errors. ## 12.0.1 - 2017-02-07 ### Fixed * Files getting truncated and edited with bad characters - issue [#626](https://github.com/datproject/dat/issues/626) and [#623](https://github.com/datproject/dat/issues/623) * Source files getting overwritten (issue [#628](https://github.com/datproject/dat/issues/628)) * Duplicate files getting imported ## 12.0.0 - 2017-02-06 Big new release! See the [release notes](https://github.com/datproject/dat/releases/tag/v12.0.0) on Github. ## 11.6.0 - 2016-11-16 ### Removed * webrtc support ### Fixed * Fail gracefully if another dat is running in directory * Handle `dat.open` errors * Progress bar incorrectly showing 100% complete and 0 bytes ### Added * Use graceful-fs to avoid EMFILE errors ## 11.5.5 - 2016-11-07 ### Fixed * Better download statistics using blocks instead of bytes * Fix share stats on resuming without file changes * Fix calculating size UI for large files ### Changed * Update status logger. Uses [ansi-diff-stream](https://github.com/mafintosh/ansi-diff-stream) for updating CLI output now. ## 11.5.4 - 2016-10-28 ### Changed * Turn off `--watchFiles` by default * Simplify progress UI ## 11.5.3 - 2016-10-28 ### Fixed * Fix `dat` command with no arguments ## 11.5.2 - 2016-10-24 ### Fixed * Fix `dat --doctor` ## 11.5.1 - 2016-10-24 ### Fixed * Resuming a folder previously shared fixed. ## 11.5.0 - 2016-10-20 ### Added * Accept dat.land links * Allow `dat ` to resume a downloaded link ### Fixed * Improved error output for incorrect params ## 11.4.0 - 2016-10-06 ### Added * `--ignore-hidden` option. Ignores hidden files by default. * `--signalhub` option to override default signalhub URL. ### Fixed * Remove headless option from electron-webrtc. It is detected for us. * `utp` is true by default ## 11.3.1 - 2016-09-21 ### Fixed * Use `--quiet` mode with `--debug` so output is easier to read. ## 11.3.0 - 2016-09-18 ### Added * `--webrtc` option. Uses electron-webrtc to run via webrtc. ## 11.2.0 - 2016-09-14 ### Added * `--temp` option. Uses memdb as database instead of `.dat` folder. * Print message when download finishes telling user they can exit. * Add option for turning off UTP * Use dat-js module (includes using hyperdrive-import-files for appending) ### Fixed * Download finished message not displayed when dat live updates * Download speed removed when download is finished ## 11.1.2 - 2016-07-18 ### Fixed * Zero bytes total when downloading Dat with single file ## 11.1.1 - 2016-07-15 ### Fixed * Create download directory if doesn't exist * Accept dat:// links for dat-desktop * Throw error when two different dats are downloaded to same folder ## 11.1.0 - 2016-07-15 ### Fixed * Use yolowatch module for recursive live updates * Improved stats for edge cases * Print link with --quiet argument * Better stat & progress output with hyperdrive/hypercore events ### Changed * Simplified and clean up CLI output * Improve modularity of library * Move logger module into own npm package, status-logger * Store key in .dat db without encoding as hex string (#498) * upgrade to hyperdrive 7 ### Removed * List download option (will be re-added pending a hyperdrive update) ### Added * Accept dat-encoding for 50 character links ## 11.0.2 - 2016-06-23 ### Fixed * Live mode with recursive adding files! ### Changed * Separate yoloWatch to module ## 11.0.1 - 2016-06-20 ### Fixed * Create download directory if it doesn't exist ### Added * Updated Docs ## 11.0.0 - 2016-06-17 ### Added * Live dat by default * Added the `.dat` folder to store metadata and keys * Resume dat share and download in existing .dat directory * Store metadata using leveldb * --list option on download to list files * --exit option on download to close process on completion ### Changed * New proposed RC2 API * --static option change to --snapshot * Use Hyperdrive-archive-swarm module for swarm ### Removed * --seed option, stays open by default now * --no-color option * --append option, append by default now ## 10.1.1 - 2016-06-09 ### Fixed * Fix file count on live share * Fix total percentage on share ## 10.1.0 - 2016-06-08 ### Changed * Show progress in CLI output ## 10.0.2 - 2016-06-07 ### Fixed * Fix --static sharing * Fix --doctor ## 10.0.1 - 2016-06-06 ### Fixed * Share argument * Argument bugs ## 10.0.0 - 2016-06-06 ### Added * Live sharing! ### Changed * Update to hyperdrive 6.0 * Update API to RC2 candidate ## 9.x.x and earlier These refer to the pre-1.0 versions of dat and are omitted. ================================================ FILE: download.sh ================================================ #!/bin/bash # gets latest dat release zip for platform and extracts runnable binary into ~/.dat # usage: wget -qO- https://raw.githubusercontent.com/datproject/dat/master/bin/install.sh | bash # based on https://github.com/jpillora/installer/blob/master/scripts/download.sh DAT_DIR="$HOME/.dat/releases" function cleanup { rm -rf $DAT_DIR/tmp.zip > /dev/null } function fail { cleanup msg=$1 echo "============" echo "Error: $msg" 1>&2 exit 1 } function install { # bash check [ ! "$BASH_VERSION" ] && fail "Please use bash instead" GET="" if which curl > /dev/null; then GET="curl" GET="$GET --fail -# -L" elif which wget > /dev/null; then GET="wget" GET="$GET -qO-" else fail "neither wget/curl are installed" fi case `uname -s` in Darwin) OS="macos";; Linux) OS="linux";; *) fail "unsupported os: $(uname -s)";; esac if uname -m | grep 64 > /dev/null; then ARCH="x64" else fail "only arch x64 is currently supported for single file install. please use npm instead. your arch is: $(uname -m)" fi echo "Fetching latest Dat release version from GitHub" LATEST=$($GET -qs https://api.github.com/repos/datproject/dat/releases/latest | grep tag_name | head -n 1 | cut -d '"' -f 4); mkdir -p $DAT_DIR || fail "Could not create directory $DAT_DIR, try manually downloading zip and extracting instead." cd $DAT_DIR RELEASE="dat-${LATEST:1}-${OS}-${ARCH}" URL="https://github.com/datproject/dat/releases/download/${LATEST}/${RELEASE}.zip" which unzip > /dev/null || fail "unzip is not installed" echo "Downloading $URL" bash -c "$GET $URL" > $DAT_DIR/tmp.zip || fail "download failed" unzip -o -qq $DAT_DIR/tmp.zip || fail "unzip failed" BIN="$DAT_DIR/$RELEASE/dat" chmod +x $BIN || fail "chmod +x failed" cleanup printf "Dat $LATEST has been downloaded successfully. Execute it with this command:\n\n${BIN}\n\nAdd it to your PATH with this command (add this to .bash_profile/.bashrc):\n\nexport PATH=\"\$PATH:$DAT_DIR/$RELEASE\"\n" } install ================================================ FILE: index.js ================================================ module.exports = require('dat-node') ================================================ FILE: package.json ================================================ { "name": "dat", "version": "14.0.3", "description": "Dat is the package manager for data. Easily share and version control data.", "keywords": [ "dat", "dat protocol", "hyperdrive", "decentralized", "file sharing" ], "main": "index.js", "bin": { "dat": "bin/cli.js" }, "scripts": { "auth-server": "DEBUG=* node scripts/auth-server.js", "install-precommit": "echo ./node_modules/.bin/standard > .git/hooks/pre-commit && chmod +x .git/hooks/pre-commit", "standard": "standard", "deps": "dependency-check . && dependency-check . --extra --no-dev", "test": "standard && npm run deps && tape 'test/*.js'", "test-only": "tape 'test/*.js'", "package": "rm -rf builds && npm run pkg && ./package.sh", "pkg": "pkg package.json -o builds/dat" }, "repository": { "type": "git", "url": "git://github.com/datproject/dat.git" }, "author": "Dat Project", "license": "BSD-3-Clause", "bugs": { "url": "https://github.com/datproject/dat/issues" }, "homepage": "https://datproject.org", "directories": { "test": "tests" }, "dependencies": { "bytes": "^3.1.0", "chalk": "^2.4.2", "cli-truncate": "^1.0.0", "dat-encoding": "^5.0.1", "dat-json": "^1.0.3", "dat-link-resolve": "^2.3.0", "dat-log": "^2.0.0", "dat-node": "^4.0.0", "dat-registry": "^4.0.1", "debug": "^4.0.0", "neat-log": "^3.1.0", "prettier-bytes": "^1.0.3", "progress-string": "^1.2.1", "prompt": "^1.0.0", "pump": "^3.0.0", "rimraf": "^2.7.1", "speedometer": "^1.1.0", "subcommand": "^2.1.1", "throttle": "^1.0.3", "update-notifier": "^2.3.0" }, "devDependencies": { "cross-zip-cli": "^1.0.0", "dependency-check": "^3.4.1", "hypercore": "^6.25.2", "mkdirp": "^0.5.4", "node-fetch": "^2.6.1", "pkg": "^4.4.4", "random-access-memory": "^3.1.1", "recursive-readdir-sync": "^1.0.6", "standard": "^12.0.0", "tape": "^4.13.2", "tape-spawn": "^1.4.2", "temporary-directory": "^1.0.2" }, "pkg": { "assets": [ "./node_modules/utp-native/prebuilds/**", "./node_modules/blake2b-wasm/blake2b.wasm", "./node_modules/siphash24/siphash24.wasm" ], "targets": [ "node10-linux-x64", "node10-macos-x64", "node10-win-x64" ] } } ================================================ FILE: package.sh ================================================ #!/usr/bin/env sh # couldnt figure out undocumented 'output template' mode for pkg so wrote this # also need to include .node files until pkg supports including them in binary NODE_ABI="node.napi" VERSION=$(node -pe "require('./package.json').version") rm -rf dist mkdir dist mkdir builds/dat-$VERSION-linux-x64 mkdir builds/dat-$VERSION-macos-x64 mkdir builds/dat-$VERSION-win-x64 mv builds/dat-linux builds/dat-$VERSION-linux-x64/dat mv builds/dat-macos builds/dat-$VERSION-macos-x64/dat mv builds/dat-win.exe builds/dat-$VERSION-win-x64/dat.exe cp node_modules/utp-native/prebuilds/linux-x64/$NODE_ABI.node builds/dat-$VERSION-linux-x64/ cp node_modules/utp-native/prebuilds/darwin-x64/$NODE_ABI.node builds/dat-$VERSION-macos-x64/ cp node_modules/utp-native/prebuilds/win32-x64/$NODE_ABI.node builds/dat-$VERSION-win-x64/ cp LICENSE builds/dat-$VERSION-linux-x64/ cp LICENSE builds/dat-$VERSION-macos-x64/ cp LICENSE builds/dat-$VERSION-win-x64/ cp README.md builds/dat-$VERSION-linux-x64/README cp README.md builds/dat-$VERSION-macos-x64/README cp README.md builds/dat-$VERSION-win-x64/README cd builds ../node_modules/.bin/cross-zip dat-$VERSION-linux-x64 ../dist/dat-$VERSION-linux-x64.zip ../node_modules/.bin/cross-zip dat-$VERSION-macos-x64 ../dist/dat-$VERSION-macos-x64.zip ../node_modules/.bin/cross-zip dat-$VERSION-win-x64 ../dist/dat-$VERSION-win-x64.zip rm -rf builds # now travis will upload the 3 zips in dist to the release ================================================ FILE: scripts/auth-server.js ================================================ var createServer = require('../tests/helpers/auth-server') createServer(process.env.PORT || 8888, function (err, server, closeServer) { if (err) throw err process.on('exit', close) process.on('SIGINT', close) function close (cb) { closeServer(function () { process.exit() }) } }) ================================================ FILE: snap/snapcraft.yaml ================================================ name: dat version: '13.11.4' summary: Share & live sync files anywhere via command line description: | Use Dat command line to share files with version control, back up data to servers, browse remote files on demand, and automate long-term data preservation. grade: 'stable' confinement: 'strict' apps: dat: command: dat plugs: - home - network - network-bind - removable-media parts: dat: source: https://github.com/datproject/dat.git source-tag: 'v13.11.4' plugin: nodejs node-engine: 10.9.0 ================================================ FILE: src/commands/auth/login.js ================================================ module.exports = { name: 'login', command: login, help: [ 'Login to a Dat registry server', 'Usage: dat login []', '', 'Publish your dats so other users can discovery them.', 'Please register before trying to login.' ].join('\n'), options: [ { name: 'server', help: 'Your Dat registry server (must be registered to login).' } ] } function login (opts) { var prompt = require('prompt') var output = require('neat-log/output') var chalk = require('chalk') var Registry = require('../../registry') if (opts._[0]) opts.server = opts._[0] var welcome = output(` Welcome to ${chalk.green(`dat`)} program! Login to publish your dats. `) console.log(welcome) var schema = { properties: { server: { description: chalk.magenta('Dat registry'), default: opts.server || 'datbase.org', required: true }, email: { description: chalk.magenta('Email'), message: 'Email required', required: true }, password: { description: chalk.magenta('Password'), message: 'Password required', required: true, hidden: true, replace: '*' } } } prompt.override = opts prompt.message = '' prompt.start() prompt.get(schema, function (err, results) { if (err) return exitErr(err) opts.server = results.server makeRequest(results) }) function makeRequest (user) { var client = Registry(opts) client.login({ email: user.email, password: user.password }, function (err, resp, body) { if (err && err.message) return exitErr(err.message) else if (err) return exitErr(err.toString()) console.log(output(` Logged you in to ${chalk.green(opts.server)}! Now you can publish dats and share: * Run ${chalk.green(`dat publish`)} to publish a dat! * View & Share your dats at ${opts.server} `)) process.exit(0) }) } } function exitErr (err) { console.error(err) process.exit(1) } ================================================ FILE: src/commands/auth/logout.js ================================================ module.exports = { name: 'logout', command: logout, help: [ 'Logout from current Dat registry server', 'Usage: dat logout []', '', 'Specify server if you want to from non-active other server.', 'Check active server with `dat whoami`.' ].join('\n'), options: [ { name: 'server', help: 'Server to log out of. Defaults to active login.' } ] } function logout (opts) { var chalk = require('chalk') var Registry = require('../../registry') if (opts._[0]) opts.server = opts._[0] var client = Registry(opts) var whoami = client.whoami() if (!whoami || !whoami.token) return exitErr('Not currently logged in to that server.') client.logout(function (err) { if (err) return exitErr(err) console.log(`Logged out of ${chalk.green(whoami.server)}`) process.exit(0) }) } function exitErr (err) { console.error(err) process.exit(1) } ================================================ FILE: src/commands/auth/register.js ================================================ module.exports = { name: 'register', command: register, help: [ 'Register with a public Dat registry', 'Usage: dat register []', '', 'Register with datbase.org or other registries to publish your dats.' ].join('\n'), options: [ { name: 'server', help: 'Your Dat registry.' } ] } function register (opts) { var prompt = require('prompt') var output = require('neat-log/output') var chalk = require('chalk') var Registry = require('../../registry') // TODO: check if logged in? if (opts._[0]) opts.server = opts._[0] var welcome = output(` Welcome to ${chalk.green(`dat`)} program! Create a new account with a Dat registry. `) console.log(welcome) var schema = { properties: { server: { description: chalk.magenta('Dat registry'), default: opts.server || 'datbase.org', required: true }, username: { description: chalk.magenta('Username'), message: 'Username required', required: true }, email: { description: chalk.magenta('Email'), message: 'Email required', required: true }, password: { description: chalk.magenta('Password'), message: 'Password required', required: true, hidden: true, replace: '*' } } } prompt.override = opts prompt.message = '' prompt.start() prompt.get(schema, function (err, results) { if (err) return exitErr(err) opts.server = results.server makeRequest(results) }) function makeRequest (user) { var client = Registry(opts) client.register({ email: user.email, username: user.username, password: user.password }, function (err) { if (err && err.message) return exitErr(err.message) else if (err) return exitErr(err.toString()) console.log(output(` Created account on ${chalk.green(opts.server)}! Login to start publishing: ${chalk.green(`dat login`)} `)) process.exit(0) }) } } function exitErr (err) { console.error(err) process.exit(1) } ================================================ FILE: src/commands/auth/whoami.js ================================================ module.exports = { name: 'whoami', command: whoami, help: [ 'Get login information', 'Usage: dat login []', '', 'Get information for active registry or specify your registry.' ].join('\n'), options: [ { name: 'server', help: 'Server to get login information for. Defaults to active login.' } ] } function whoami (opts) { var output = require('neat-log/output') var chalk = require('chalk') var Registry = require('../../registry') if (opts._[0]) opts.server = opts._[0] var client = Registry(opts) var login = client.whoami() if (!login || !login.token) { if (!opts.server) return exitErr('No login information found.') return exitErr('No login information found for that server.') } console.log(output(` Your active Dat registry information: --- ${chalk.green(login.server)} Email: ${login.email} Username: ${login.username} --- Change your registry by logging in again: ${chalk.dim.green('dat login ')} `)) process.exit(0) } function exitErr (err) { console.error(err) process.exit(1) } ================================================ FILE: src/commands/clone.js ================================================ module.exports = { name: 'clone', command: clone, help: [ 'Clone a remote Dat archive', '', 'Usage: dat clone [download-folder]' ].join('\n'), options: [ { name: 'empty', boolean: false, default: false, help: 'Do not download files by default. Files must be synced manually.' }, { name: 'upload', boolean: true, default: true, help: 'announce your address on link (improves connection capability) and upload data to other downloaders.' }, { name: 'show-key', boolean: true, default: false, abbr: 'k', help: 'print out the dat key' } ] } function clone (opts) { var fs = require('fs') var path = require('path') var rimraf = require('rimraf') var Dat = require('dat-node') var linkResolve = require('dat-link-resolve') var neatLog = require('neat-log') var archiveUI = require('../ui/archive') var trackArchive = require('../lib/archive') var discoveryExit = require('../lib/discovery-exit') var onExit = require('../lib/exit') var parseArgs = require('../parse-args') var debug = require('debug')('dat') var parsed = parseArgs(opts) opts.key = parsed.key || opts._[0] // pass other links to resolver opts.dir = parsed.dir opts.showKey = opts['show-key'] // using abbr in option makes printed help confusing opts.sparse = opts.empty debug('clone()') // cmd: dat /path/to/dat.json (opts.key is path to dat.json) if (fs.existsSync(opts.key)) { try { opts.key = getDatJsonKey() } catch (e) { debug('error reading dat.json key', e) } } debug(Object.assign({}, opts, { key: '', _: null })) // don't show key var neat = neatLog(archiveUI, { logspeed: opts.logspeed, quiet: opts.quiet, debug: opts.debug }) neat.use(trackArchive) neat.use(discoveryExit) neat.use(onExit) neat.use(function (state, bus) { if (!opts.key) return bus.emit('exit:warn', 'key required to clone') state.opts = opts var createdDirectory = null // so we can delete directory if we get error // Force these options for clone command opts.exit = (opts.exit !== false) // opts.errorIfExists = true // TODO: do we want to force this? linkResolve(opts.key, function (err, key) { if (err && err.message.indexOf('Invalid key') === -1) return bus.emit('exit:error', 'Could not resolve link') else if (err) return bus.emit('exit:warn', 'Link is not a valid Dat link.') opts.key = key createDir(opts.key, function () { bus.emit('key', key) runDat() }) }) function createDir (key, cb) { debug('Checking directory for clone') // Create the directory if it doesn't exist // If no dir is specified, we put dat in a dir with name = key if (!opts.dir) opts.dir = key if (!Buffer.isBuffer(opts.dir) && typeof opts.dir !== 'string') { return bus.emit('exit:error', 'Directory path must be a string or Buffer') } fs.access(opts.dir, fs.F_OK, function (err) { if (!err) { createdDirectory = false return cb() } debug('No existing directory, creating it.') createdDirectory = true fs.mkdir(opts.dir, cb) }) } function runDat () { Dat(opts.dir, opts, function (err, dat) { if (err && err.name === 'ExistsError') return bus.emit('exit:warn', 'Existing archive in this directory. Use pull or sync to update.') if (err) { if (createdDirectory) rimraf.sync(dat.path) return bus.emit('exit:error', err) } if (dat.writable) return bus.emit('exit:warn', 'Archive is writable. Cannot clone your own archive =).') state.dat = dat state.title = 'Cloning' bus.emit('dat') bus.emit('render') }) } }) function getDatJsonKey () { var datPath = opts.key var stat = fs.lstatSync(datPath) if (stat.isDirectory()) datPath = path.join(datPath, 'dat.json') if (!fs.existsSync(datPath) || path.basename(datPath) !== 'dat.json') { if (stat.isFile()) throw new Error('must specify existing dat.json file to read key') throw new Error('directory must contain a dat.json') } debug('reading key from dat.json:', datPath) return JSON.parse(fs.readFileSync(datPath, 'utf8')).url } } ================================================ FILE: src/commands/create.js ================================================ module.exports = { name: 'create', command: create, help: [ 'Create an empty dat and dat.json', '', 'Usage: dat create [directory]' ].join('\n'), options: [ { name: 'yes', boolean: true, default: false, abbr: 'y', help: 'Skip dat.json creation.' }, { name: 'title', help: 'the title property for dat.json' }, { name: 'description', help: 'the description property for dat.json' } ] } function create (opts) { var path = require('path') var fs = require('fs') var Dat = require('dat-node') var output = require('neat-log/output') var DatJson = require('dat-json') var prompt = require('prompt') var chalk = require('chalk') var parseArgs = require('../parse-args') var debug = require('debug')('dat') debug('dat create') if (!opts.dir) { opts.dir = parseArgs(opts).dir || process.cwd() } var welcome = `Welcome to ${chalk.green(`dat`)} program!` var intro = output(` You can turn any folder on your computer into a Dat. A Dat is a folder with some magic. Your dat is ready! We will walk you through creating a 'dat.json' file. (You can skip dat.json and get started now.) Learn more about dat.json: ${chalk.blue(`https://github.com/datprotocol/dat.json`)} ${chalk.dim('Ctrl+C to exit at any time')} `) var outro // Force certain options opts.errorIfExists = true console.log(welcome) Dat(opts.dir, opts, function (err, dat) { if (err && err.name === 'ExistsError') return exitErr('\nArchive already exists.\nYou can use `dat sync` to update.') if (err) return exitErr(err) outro = output(` Created empty Dat in ${dat.path}/.dat Now you can add files and share: * Run ${chalk.green(`dat share`)} to create metadata and sync. * Copy the unique dat link and securely share it. ${chalk.blue(`dat://${dat.key.toString('hex')}`)} `) if (opts.yes) return done() console.log(intro) var datjson = DatJson(dat.archive, { file: path.join(opts.dir, 'dat.json') }) fs.readFile(path.join(opts.dir, 'dat.json'), 'utf-8', function (err, data) { if (err || !data) return doPrompt() data = JSON.parse(data) debug('read existing dat.json data', data) doPrompt(data) }) function doPrompt (data) { if (!data) data = {} var schema = { properties: { title: { description: chalk.magenta('Title'), default: data.title || '', // pattern: /^[a-zA-Z\s\-]+$/, // message: 'Name must be only letters, spaces, or dashes', required: false }, description: { description: chalk.magenta('Description'), default: data.description || '' } } } prompt.override = { title: opts.title, description: opts.description } prompt.message = '' // chalk.green('> ') // prompt.delimiter = '' prompt.start() prompt.get(schema, writeDatJson) function writeDatJson (err, results) { if (err) return exitErr(err) // prompt error if (!results.title && !results.description) return done() datjson.create(results, done) } } function done (err) { if (err) return exitErr(err) console.log(outro) } }) function exitErr (err) { if (err && err.message === 'canceled') { console.log('') console.log(outro) process.exit(0) } console.error(err) process.exit(1) } } ================================================ FILE: src/commands/doctor.js ================================================ module.exports = { name: 'doctor', help: [ 'Call the Doctor! Runs two tests:', ' 1. Check if you can connect to a peer on a public server.', ' 2. Gives you a link to test direct peer connections.', '', 'Usage: dat doctor []' ].join('\n'), options: [], command: function (opts) { var doctor = require('dat-doctor') opts.peerId = opts._[0] doctor(opts) } } ================================================ FILE: src/commands/keys.js ================================================ module.exports = { name: 'keys', command: keys, help: [ 'View & manage dat keys', '', 'Usage:', '', ' dat keys view dat key and discovery key', ' dat keys export export dat secret key', ' dat keys import import dat secret key to make a dat writable', '' ].join('\n'), options: [ { name: 'discovery', boolean: true, default: false, help: 'Print Discovery Key' } ] } function keys (opts) { var Dat = require('dat-node') var parseArgs = require('../parse-args') var debug = require('debug')('dat') debug('dat keys') if (!opts.dir) { opts.dir = parseArgs(opts).dir || process.cwd() } opts.createIfMissing = false // keys must always be a resumed archive Dat(opts.dir, opts, function (err, dat) { if (err && err.name === 'MissingError') return exit('Sorry, could not find a dat in this directory.') if (err) return exit(err) run(dat, opts) }) } function run (dat, opts) { var subcommand = require('subcommand') var prompt = require('prompt') var config = { root: { command: function () { console.log(`dat://${dat.key.toString('hex')}`) if (opts.discovery) console.log(`Discovery key: ${dat.archive.discoveryKey.toString('hex')}`) process.exit() } }, commands: [ { name: 'export', command: function foo (args) { if (!dat.writable) return exit('Dat must be writable to export.') console.log(dat.archive.metadata.secretKey.toString('hex')) } }, { name: 'import', command: function bar (args) { if (dat.writable) return exit('Dat is already writable.') importKey() } } ] } subcommand(config)(process.argv.slice(3)) function importKey () { // get secret key & write var schema = { properties: { key: { pattern: /^[a-z0-9]{128}$/, message: 'Use `dat keys export` to get the secret key (128 character hash).', hidden: true, required: true, description: 'dat secret key' } } } prompt.message = '' prompt.start() prompt.get(schema, function (err, data) { if (err) return done(err) var secretKey = data.key if (typeof secretKey === 'string') secretKey = Buffer.from(secretKey, 'hex') // Automatically writes the metadata.ogd file dat.archive.metadata._storage.secretKey.write(0, secretKey, done) }) function done (err) { if (err) return exit(err) console.log('Successful import. Dat is now writable.') exit() } } } function exit (err) { if (err) { console.error(err) process.exit(1) } process.exit(0) } ================================================ FILE: src/commands/log.js ================================================ module.exports = { name: 'log', help: [ 'View history and information about a dat', '', 'Usage: dat log [dir|link]' ].join('\n'), options: [ { name: 'live', boolean: true, default: false, help: 'View live updates to history.' } ], command: function (opts) { var log = require('dat-log') log(opts) } } ================================================ FILE: src/commands/publish.js ================================================ module.exports = { name: 'publish', command: publish, help: [ 'Publish your dat to a Dat registry', 'Usage: dat publish []', '', 'By default it will publish to your active registry.', 'Specify the server to change where the dat is published.' ].join('\n'), options: [ { name: 'server', help: 'Publish dat to this registry. Defaults to active login.' } ] } function publish (opts) { var path = require('path') var Dat = require('dat-node') var encoding = require('dat-encoding') var output = require('neat-log/output') var prompt = require('prompt') var chalk = require('chalk') var DatJson = require('dat-json') var xtend = Object.assign var Registry = require('../registry') if (!opts.dir) opts.dir = process.cwd() if (opts._[0]) opts.server = opts._[0] if (!opts.server) opts.server = 'datbase.org' // nicer error message if not logged in var client = Registry(opts) var whoami = client.whoami() if (!whoami || !whoami.token) { var loginErr = output(` Welcome to ${chalk.green(`dat`)} program! Publish your dats to ${chalk.green(opts.server)}. ${chalk.bold('Please login before publishing')} ${chalk.green('dat login')} New to ${chalk.green(opts.server)} and need an account? ${chalk.green('dat register')} Explore public dats at ${chalk.blue('datbase.org/explore')} `) return exitErr(loginErr) } opts.createIfMissing = false // publish must always be a resumed archive Dat(opts.dir, opts, function (err, dat) { if (err && err.name === 'MissingError') return exitErr('No existing dat in this directory. Create a dat before publishing.') else if (err) return exitErr(err) dat.joinNetwork() // join network to upload metadata var datjson = DatJson(dat.archive, { file: path.join(dat.path, 'dat.json') }) datjson.read(publish) function publish (_, data) { // ignore datjson.read() err, we'll prompt for name // xtend dat.json with opts var datInfo = xtend({ name: opts.name, url: 'dat://' + encoding.toStr(dat.key), // force correct url in publish? what about non-dat urls? title: opts.title, description: opts.description }, data) var welcome = output(` Publishing dat to ${chalk.green(opts.server)}! `) console.log(welcome) if (datInfo.name) return makeRequest(datInfo) prompt.message = '' prompt.start() prompt.get({ properties: { name: { description: chalk.magenta('dat name'), pattern: /^[a-zA-Z0-9-]+$/, message: `A dat name can only have letters, numbers, or dashes.\n Like ${chalk.bold('cool-cats-12meow')}`, required: true } } }, function (err, results) { if (err) return exitErr(err) datInfo.name = results.name makeRequest(datInfo) }) } function makeRequest (datInfo) { console.log(`Please wait, '${chalk.bold(datInfo.name)}' will soon be ready for its great unveiling...`) client.dats.create(datInfo, function (err, resp, body) { if (err) { if (err.message) { if (err.message === 'timed out') { return exitErr(output(`${chalk.red('\nERROR: ' + opts.server + ' could not connect to your computer.')} Troubleshoot here: ${chalk.green('https://docs.datproject.org/troubleshooting#networking-issues')} `)) } var str = err.message.trim() if (str === 'jwt expired') return exitErr(`Session expired, please ${chalk.green('dat login')} again`) return exitErr('ERROR: ' + err.message) // node error } // server response errors return exitErr('ERROR: ' + err.toString()) } if (body.statusCode === 400) return exitErr(new Error(body.message)) datjson.write(datInfo, function (err) { if (err) return exitErr(err) // TODO: write published url to dat.json (need spec) var msg = output(` We ${body.updated === 1 ? 'updated' : 'published'} your dat! ${chalk.blue.underline(`${opts.server}/${whoami.username}/${datInfo.name}`)} `)// TODO: get url back? it'd be better to confirm link than guess username/datname structure console.log(msg) if (body.updated === 1) { console.log(output(` ${chalk.dim.green('Cool fact #21')} ${opts.server} will live update when you are sharing your dat! You only need to publish again if your dat link changes. `)) } else { console.log(output(` Remember to use ${chalk.green('dat share')} before sharing. This will make sure your dat is available. `)) } process.exit(0) }) }) } }) } function exitErr (err) { console.error(err) process.exit(1) } ================================================ FILE: src/commands/pull.js ================================================ module.exports = { name: 'pull', command: pull, help: [ 'Pull updates from a cloned Dat archive', '', 'Usage: dat pull' ].join('\n'), options: [ { name: 'exit', boolean: false, help: 'exit after specified number of seconds, to give the dat network time to find updates. (default: 12)' }, { name: 'upload', boolean: true, default: true, help: 'announce your address on link (improves connection capability) and upload data to other downloaders.' }, { name: 'selectFromFile', boolean: false, default: '.datdownload', help: 'Sync only the list of selected files or directories in the given file.', abbr: 'select-from-file' }, { name: 'select', boolean: false, default: false, help: 'Sync only the list of selected files or directories.' }, { name: 'show-key', boolean: true, default: false, abbr: 'k', help: 'print out the dat key' } ] } function pull (opts) { var Dat = require('dat-node') var neatLog = require('neat-log') var archiveUI = require('../ui/archive') var trackArchive = require('../lib/archive') var selectiveSync = require('../lib/selective-sync') var discoveryExit = require('../lib/discovery-exit') var onExit = require('../lib/exit') var parseArgs = require('../parse-args') var debug = require('debug')('dat') debug('dat pull') if (!opts.dir) { var parsed = parseArgs(opts) opts.key = parsed.key opts.dir = parsed.dir || process.cwd() } opts.showKey = opts['show-key'] // using abbr in option makes printed help confusing // Force these options for pull command opts.createIfMissing = false // If --exit is specified without a number of seconds, default to 12 if (opts.exit) { opts.exit = typeof opts.exit === 'number' ? opts.exit : 12 } var neat = neatLog(archiveUI, { logspeed: opts.logspeed, quiet: opts.quiet, debug: opts.debug }) neat.use(trackArchive) neat.use(discoveryExit) neat.use(onExit) neat.use(function (state, bus) { state.opts = opts selectiveSync(state, opts) Dat(opts.dir, opts, function (err, dat) { if (err && err.name === 'MissingError') return bus.emit('exit:warn', 'No existing archive in this directory. Use clone to download a new archive.') if (err) return bus.emit('exit:error', err) if (dat.writable) return bus.emit('exit:warn', 'Archive is writable. Cannot pull your own archive.') state.dat = dat bus.emit('dat') bus.emit('render') }) }) } ================================================ FILE: src/commands/status.js ================================================ module.exports = { name: 'status', command: status, help: [ 'Get information on about the Dat in a directory.', '', 'Usage: dat status' ].join('\n'), options: [] } function status (opts) { var Dat = require('dat-node') var neatLog = require('neat-log') var statusUI = require('../ui/status') var onExit = require('../lib/exit') var parseArgs = require('../parse-args') var debug = require('debug')('dat') debug('dat status') if (!opts.dir) { opts.dir = parseArgs(opts).dir || process.cwd() } opts.createIfMissing = false // sync must always be a resumed archive var neat = neatLog(statusUI, { logspeed: opts.logspeed, quiet: opts.quiet, debug: opts.debug }) neat.use(onExit) neat.use(function (state, bus) { state.opts = opts Dat(opts.dir, opts, function (err, dat) { if (err && err.name === 'MissingError') return bus.emit('exit:warn', 'Sorry, could not find a dat in this directory.') if (err) return bus.emit('exit:error', err) state.dat = dat var stats = dat.trackStats() if (stats.get().version === dat.version) return exit() stats.on('update', function () { if (stats.get().version === dat.version) return exit() }) function exit () { bus.render() process.exit(0) } }) }) } ================================================ FILE: src/commands/sync.js ================================================ module.exports = { name: 'sync', command: sync, help: [ 'Sync a Dat archive with the network', 'Watch and import file changes', '', 'Usage: dat sync' ].join('\n'), options: [ { name: 'import', boolean: true, default: true, help: 'Import files from the directory to the database (when writable).' }, { name: 'ignoreHidden', boolean: true, default: true, abbr: 'ignore-hidden' }, { name: 'selectFromFile', boolean: false, default: '.datdownload', help: 'Sync only the list of selected files or directories in the given file.', abbr: 'select-from-file' }, { name: 'select', boolean: false, default: false, help: 'Sync only the list of selected files or directories.' }, { name: 'watch', boolean: true, default: true, help: 'Watch for changes and import updated files (Dat Writable).' }, { name: 'show-key', boolean: true, default: true, abbr: 'k', help: 'Print out the dat key.' } ] } function sync (opts) { var Dat = require('dat-node') var neatLog = require('neat-log') var archiveUI = require('../ui/archive') var selectiveSync = require('../lib/selective-sync') var trackArchive = require('../lib/archive') var onExit = require('../lib/exit') var parseArgs = require('../parse-args') var debug = require('debug')('dat') debug('dat sync') var parsed = parseArgs(opts) opts.key = parsed.key opts.dir = parsed.dir || process.cwd() opts.showKey = opts['show-key'] // using abbr in option makes printed help confusing // TODO: if dat-store running, add this dat to the local store and then exit = true opts.exit = false var neat = neatLog(archiveUI, { logspeed: opts.logspeed, quiet: opts.quiet, debug: opts.debug }) neat.use(trackArchive) neat.use(onExit) neat.use(function (state, bus) { state.opts = opts selectiveSync(state, opts) Dat(opts.dir, opts, function (err, dat) { if (err && err.name === 'IncompatibleError') return bus.emit('exit:warn', 'Directory contains incompatible dat metadata. Please remove the .dat folder in this directory.') if (err) return bus.emit('exit:error', err) state.dat = dat bus.emit('dat') bus.emit('render') }) }) } ================================================ FILE: src/commands/unpublish.js ================================================ module.exports = { name: 'unpublish', command: unpublish, options: [ { name: 'server', help: 'Unpublish dat from this Registry.' }, { name: 'confirm', default: false, boolean: true, abbr: 'y', help: 'Confirm you want to unpublish' } ] } function unpublish (opts) { var prompt = require('prompt') var path = require('path') var Dat = require('dat-node') var output = require('neat-log/output') var chalk = require('chalk') var DatJson = require('dat-json') var Registry = require('../registry') if (opts._[0]) opts.server = opts._[0] if (!opts.dir) opts.dir = process.cwd() // run in dir for `dat unpublish` var client = Registry(opts) var whoami = client.whoami() if (!whoami || !whoami.token) { var loginErr = output(` Welcome to ${chalk.green(`dat`)} program! ${chalk.bold('You must login before unpublishing.')} ${chalk.green('dat login')} `) return exitErr(loginErr) } opts.createIfMissing = false // unpublish dont try to create new one Dat(opts.dir, opts, function (err, dat) { if (err) return exitErr(err) // TODO better error msg for non-existing archive if (!dat.writable) return exitErr('Sorry, you can only publish a dat that you created.') var datjson = DatJson(dat.archive, { file: path.join(dat.path, 'dat.json') }) datjson.read(function (err, data) { if (err) return exitErr(err) if (!data.name) return exitErr('Try `dat unpublish ` with this dat, we are having trouble reading it.') confirm(data.name) }) }) function confirm (name) { console.log(`Unpublishing '${chalk.bold(name)}' from ${chalk.green(whoami.server)}.`) prompt.message = '' prompt.colors = false prompt.start() prompt.get([{ name: 'sure', description: 'Are you sure? This cannot be undone. [y/n]', pattern: /^[a-zA-Z\s-]+$/, message: '', required: true }], function (err, results) { if (err) return console.log(err.message) if (results.sure === 'yes' || results.sure === 'y') makeRequest(name) else exitErr('Cancelled.') }) } function makeRequest (name) { client.dats.delete({ name: name }, function (err, resp, body) { if (err && err.message) exitErr(err.message) else if (err) exitErr(err.toString()) if (body.statusCode === 400) return exitErr(new Error(body.message)) console.log(`Removed your dat from ${whoami.server}`) process.exit(0) }) } } function exitErr (err) { console.error(err) process.exit(1) } ================================================ FILE: src/extensions.js ================================================ var debug = require('debug')('dat') var os = require('os') module.exports = runExtension function runExtension (opts) { debug('Trying Extenion', opts._[0]) var extName = opts._.shift() trySpawn(function () { console.error('We could not run the extension. Please make sure it is installed:') console.error(`npm install -g dat-${extName}`) process.exit(1) }) function trySpawn (cb) { var spawn = require('child_process').spawn var name = 'dat-' + extName if (os.platform() === 'win32') { name += '.cmd' } var child = spawn(name, process.argv.splice(3)) child.stdout.pipe(process.stdout) child.stderr.pipe(process.stderr) child.on('error', function (err) { if (err.code === 'ENOENT') return cb() throw err }) child.on('close', function (code) { process.exit(code) }) } } ================================================ FILE: src/lib/archive.js ================================================ var debug = require('debug')('dat') var path = require('path') var EventEmitter = require('events').EventEmitter var doImport = require('./import-progress') var stats = require('./stats') var network = require('./network') var download = require('./download') var serve = require('./serve-http') module.exports = function (state, bus) { state.warnings = state.warnings || [] bus.once('dat', function () { state.writable = state.dat.writable state.joinNetwork = !(state.joinNetwork === false) stats(state, bus) if (state.joinNetwork) network(state, bus) if (state.opts.http) serve(state, bus) if (state.writable && state.opts.import) doImport(state, bus) else if (state.opts.sparse) selectiveSync(state, bus) else download(state, bus) if (state.dat.archive.content) return bus.emit('archive:content') state.dat.archive.once('content', function () { bus.emit('archive:content') }) }) bus.once('archive:content', function () { state.hasContent = true }) } function selectiveSync (state, bus) { var archive = state.dat.archive debug('sparse mode. downloading metadata') var emitter = new EventEmitter() function download (entry) { debug('selected', entry) archive.stat(entry, function (err, stat) { if (err) return state.warnings.push(err.message) if (stat.isDirectory()) downloadDir(entry, stat) if (stat.isFile()) downloadFile(entry, stat) }) } function downloadDir (dirname, stat) { debug('downloading dir', dirname) archive.readdir(dirname, function (err, entries) { if (err) return bus.emit('exit:error', err) entries.forEach(function (entry) { emitter.emit('download', path.join(dirname, entry)) }) }) } function downloadFile (entry, stat) { var start = stat.offset var end = stat.offset + stat.blocks state.selectedByteLength += stat.size bus.emit('render') if (start === 0 && end === 0) return debug('downloading', entry, start, end) archive.content.download({ start, end }, function () { debug('success', entry) }) } emitter.on('download', download) if (state.opts.selectedFiles) state.opts.selectedFiles.forEach(download) if (state.opts.empty) { archive.metadata.update(function () { return bus.emit('exit:warn', `Dat successfully created in empty mode. Download files using pull or sync.`) }) } archive.on('update', function () { debug('archive update') bus.emit('render') }) } ================================================ FILE: src/lib/discovery-exit.js ================================================ var output = require('neat-log/output') module.exports = discoveryExit function discoveryExit (state, bus) { bus.once('network:callback', checkExit) function checkExit () { if (state.dat.network.connections || !state.opts.exit) return if (state.dat.network.connecting) return setTimeout(checkExit, 500) // wait to see if any connections resolve var msg = output(` Dat could not find any connections for that link. There may not be any sources online. Ensure that everyone is using the latest version, using dat -v `) bus.emit('exit:warn', msg) } } ================================================ FILE: src/lib/download.js ================================================ var debug = require('debug')('dat') var xtend = Object.assign module.exports = trackDownload function trackDownload (state, bus) { if (state.hasContent) return track() bus.once('archive:content', track) function track () { var archive = state.dat.archive state.download = xtend({ modified: false, nsync: false }, {}) archive.content.on('clear', function () { debug('archive clear') state.download.modified = true }) archive.content.on('download', function (index, data) { state.download.modified = true }) archive.on('syncing', function () { debug('archive syncing') state.download.nsync = false }) archive.on('sync', function () { debug('archive sync', state.stats.get()) state.download.nsync = true // if we are supposed to exit, do so if we've pulled changes or have given the network the desired wait time if (state.opts.exit) { if (state.download.modified) { return exit() } else { var delayInMilliseconds = 1000 * state.opts.exit setTimeout(exit, delayInMilliseconds) } } if (state.dat.archive.version === 0) { // TODO: deal with this. // Sync sometimes fires early when it should wait for update. } bus.emit('render') }) archive.on('update', function () { debug('archive update') bus.emit('render') }) function exit () { if (state.stats.get().version !== archive.version) { return state.stats.on('update', exit) } state.exiting = true bus.render() process.exit(0) } } } ================================================ FILE: src/lib/exit.js ================================================ module.exports = onExit function onExit (state, bus) { bus.on('exit:error', onError) bus.on('exit:warn', function (err) { onError(err, true) }) bus.on('exit', function () { state.exiting = true bus.render() process.exit() }) function onError (err, clear) { if (clear) bus.clear() console.error(err) process.exit(1) } } ================================================ FILE: src/lib/import-progress.js ================================================ var xtend = Object.assign module.exports = trackImport function trackImport (state, bus) { if (state.dat) return track() bus.once('dat', track) function track () { var progress = state.dat.importFiles(state.opts, function (err) { if (err) return bus.emit('exit:error', err) state.importer.fileImport = null state.exiting = true bus.emit('render') }) state.importer = xtend({ importedBytes: 0, count: progress.count, liveImports: [], indexSpeed: progress.indexSpeed }, progress) bus.emit('dat:importer') var counting = setInterval(function () { // Update file count in progress counting (for big dirs) bus.emit('render') }, state.opts.logspeed) progress.on('count', function (count) { clearInterval(counting) state.count = count state.count.done = true bus.emit('render') }) progress.on('del', function (src, dst) { if (src.live) state.importer.liveImports.push({ src: src, dst: dst, type: 'del' }) }) progress.on('put', function (src, dst) { if (src.live) state.importer.liveImports.push({ src: src, dst: dst, type: 'put' }) if (src.stat.isDirectory()) return state.importer.fileImport = { src: src, dst: dst, progress: 0, type: 'put' } bus.emit('render') }) progress.on('put-data', function (chunk, src, dst) { state.importer.fileImport.progress += chunk.length if (!src.live) state.importer.importedBytes += chunk.length // don't include live in total state.importer.indexSpeed = progress.indexSpeed bus.emit('render') }) } } ================================================ FILE: src/lib/network.js ================================================ var bytes = require('bytes').parse var speed = require('speedometer') var throttle = require('throttle') var pump = require('pump') var debug = require('debug')('dat') var xtend = Object.assign module.exports = trackNetwork function trackNetwork (state, bus) { if (state.dat) return track() bus.once('dat', track) function track () { var opts = state.opts if (state.opts.up || state.opts.down) { opts = xtend({}, opts, { connect: function (local, remote) { var streams = [local, remote, local] if (state.opts.up) streams.splice(1, 0, throttle(bytes(state.opts.up))) if (state.opts.down) streams.splice(-1, 0, throttle(bytes(state.opts.down))) pump(streams) } }) } var network = state.dat.joinNetwork(opts, function () { bus.emit('network:callback') }) network.on('error', function (err) { if (err.code === 'EADDRINUSE') { if (opts.port) { bus.emit('exit:warn', `Specified port (${opts.port}) in use. Please use another port.`) } else { debug(err.message + ' trying random port') } } else { debug('network error:', err.message) // TODO return bus.emit('exit:error', err) } }) state.network = xtend(network, state.network) bus.emit('dat:network') network.on('connection', function (conn, info) { bus.emit('render') conn.on('close', function () { bus.emit('render') }) }) if (state.opts.sources) trackSources() if (state.stats) return trackSpeed() bus.once('dat:stats', trackSpeed) function trackSpeed () { setInterval(function () { bus.emit('render') }, state.opts.logspeed) } function trackSources () { state.sources = state.sources || {} network.on('connection', function (conn, info) { var id = info.id.toString('hex') var peerSpeed = speed() state.sources[id] = info state.sources[id].speed = peerSpeed() state.sources[id].getProgress = function () { // TODO: how to get right peer from archive.content? // var remote = conn.feeds[1].remoteLength // // state.dat.archive.content.sources[0].feed.id.toString('hex') // if (!remote) return // return remote / dat.archive.content.length } conn.feeds.map(function (feed) { feed.stream.on('data', function (data) { state.sources[id].speed = peerSpeed(data.length) bus.emit('render') }) feed.stream.on('error', function (err) { state.sources[id].error = err }) }) bus.emit('render') conn.on('close', function () { state.sources[id].speed = 0 state.sources[id].closed = true bus.emit('render') }) }) } } } ================================================ FILE: src/lib/selective-sync.js ================================================ var fs = require('fs') var path = require('path') module.exports = function (state, opts) { // selective sync stuff var parsing = opts.selectFromFile !== '.datdownload' ? opts.selectFromFile : path.join(opts.dir, '.datdownload') opts.selectedFiles = parseFiles(parsing) if (opts.select && typeof opts.select === 'string') opts.selectedFiles = opts.select.split(',') if (opts.selectedFiles) { state.title = 'Syncing' state.selectedByteLength = 0 opts.sparse = true } return state } function parseFiles (input) { var parsed = null try { if (fs.statSync(input).isFile()) { parsed = fs.readFileSync(input).toString().trim().split(/\r?\n/) } } catch (err) { if (err && !err.name === 'ENOENT') { console.error(err) process.exit(1) } } return parsed } ================================================ FILE: src/lib/serve-http.js ================================================ module.exports = runHttp function runHttp (state, bus) { if (state.dat) return serve() bus.once('dat', serve) function serve () { var port = (typeof state.opts.http === 'boolean') ? 8080 : state.opts.http var server = state.dat.serveHttp({ port: port }) server.on('listening', function () { state.http = { port: port, listening: true } bus.emit('render') }) } } ================================================ FILE: src/lib/stats.js ================================================ var xtend = Object.assign module.exports = trackStats function trackStats (state, bus) { if (state.dat) return track() bus.once('dat', track) function track () { var stats = state.dat.trackStats(state.opts) state.stats = xtend(stats, state.stats) stats.on('update', function () { bus.emit('stats:update') bus.emit('render') }) bus.emit('stats') } } ================================================ FILE: src/parse-args.js ================================================ var fs = require('fs') var path = require('path') var encoding = require('dat-encoding') module.exports = function (opts) { // dat [] arg1 arg2 [options] // parse args without options from opts._ // return parsed { dir, key } var parsed = { key: opts.key || null, dir: opts.dir || null // process.cwd() ? } // dat [] if (!opts._.length) return parsed // dat [] arg1 arg2 // arg1 = key // arg2 = dir if (opts._.length === 2) { parsed.key = opts._[0] parsed.dir = opts._[1] return parsed } // dat [] arg // arg = dir or key // First, check if key try { parsed.key = encoding.toStr(opts._[0]) return parsed } catch (err) { if (err && err.message !== 'Invalid key') { // catch non-key errors console.error(err) process.exit(1) } } try { var stat = fs.statSync(opts._[0]) if (stat.isFile()) { parsed.dir = path.resolve(path.dirname(opts._[0])) } else { parsed.dir = opts._[0] } } catch (err) { if (err && !err.name === 'ENOENT') { console.error(err) process.exit(1) } } return parsed } ================================================ FILE: src/registry.js ================================================ var xtend = Object.assign var RegistryClient = require('dat-registry') module.exports = function (opts) { var townshipOpts = { server: opts.server, config: { filepath: opts.config // defaults to ~/.datrc via dat-registry } } var defaults = { // xtend doesn't overwrite when key is present but undefined // If we want a default, make sure it's not going to passed as undefined } var options = xtend(defaults, townshipOpts) return RegistryClient(options) } ================================================ FILE: src/ui/archive.js ================================================ var path = require('path') var output = require('neat-log/output') var pretty = require('prettier-bytes') var chalk = require('chalk') var downloadUI = require('./components/download') var importUI = require('./components/import-progress') var warningsUI = require('./components/warnings') var networkUI = require('./components/network') var sourcesUI = require('./components/sources') var keyEl = require('./elements/key') var pluralize = require('./elements/pluralize') var version = require('./elements/version') var pkg = require('../../package.json') module.exports = archiveUI function archiveUI (state) { if (!state.dat) return 'Starting Dat program...' if (!state.writable && !state.hasContent) return 'Connecting to dat network...' if (!state.warnings) state.warnings = [] var dat = state.dat var stats = dat.stats.get() var title = (state.dat.resumed) ? '' : `Created new dat in ${dat.path}${path.sep}.dat\n` var progressView if (state.writable || state.opts.showKey) { title += `${keyEl(dat.key)}\n` } if (state.title) title += state.title else if (state.writable) title += 'Sharing dat' else title += 'Downloading dat' if (state.opts.sparse) title += `: ${state.opts.selectedFiles.length} ${pluralize('file', state.opts.selectedFiles.length)} (${pretty(state.selectedByteLength)})` else if (stats.version > 0) title += `: ${stats.files} ${pluralize('file', stats.file)} (${pretty(stats.byteLength)})` else if (stats.version === 0) title += ': (empty archive)' if (state.http && state.http.listening) title += `\nServing files over http at http://localhost:${state.http.port}` if (!state.writable) { progressView = downloadUI(state) } else { if (state.opts.import) { progressView = importUI(state) } else { progressView = 'Not importing files.' // TODO: ? } } return output(` ${version(pkg.version)} ${title} ${state.joinNetwork ? '\n' + networkUI(state) : ''} ${progressView} ${state.opts.sources ? sourcesUI(state) : ''} ${state.warnings ? warningsUI(state) : ''} ${state.exiting ? 'Exiting the Dat program...' : chalk.dim('Ctrl+C to Exit')} `) } ================================================ FILE: src/ui/components/download.js ================================================ var output = require('neat-log/output') var bar = require('progress-string') module.exports = networkUI function networkUI (state) { var stats = state.stats.get() var download = state.download if (!stats || !download) return '' var title = 'Downloading updates...' var downBar = makeBar() if (download.nsync) { if (state.opts.exit && state.dat.archive.version === 0) { return 'dat synced. There is no content in this archive.' } if (state.opts.exit && download.modified) { return `dat sync complete.\nVersion ${stats.version}` } if (!download.modified && state.opts.exit) { title = `dat already in sync, waiting for updates.` } else { title = `dat synced, waiting for updates.` } } if (typeof state.opts.exit === 'number') { title = `dat synced, exiting in ${state.opts.exit} seconds.` } if (!stats.downloaded || !stats.length) { return '' // no metadata yet } return output(` ${title} ${downBar(stats.downloaded)} `) function makeBar () { var total = stats.length return bar({ total: total, style: function (a, b) { return `[${a}${b}] ${(100 * stats.downloaded / total).toFixed(2)}%` } }) } } ================================================ FILE: src/ui/components/import-progress.js ================================================ var output = require('neat-log/output') var pretty = require('prettier-bytes') var bar = require('progress-string') var cliTruncate = require('cli-truncate') module.exports = importUI function importUI (state) { var watch = state.opts.watch var importState = state.importer var indexSpeed = importState.indexSpeed ? `(${pretty(importState.indexSpeed)}/s)` : '' if (importState.count && !importState.count.done) { // dry run in progress if (!importState.count.files) return 'Checking for file updates...' return output(` Metadata created for ${importState.putDone.files} of ${importState.count.files} files ${indexSpeed} (Calculating file count...) ${fileImport(importState.fileImport)} `) } else if (importState.putDone.files >= importState.count.files) { // Initial import done if (!watch) return 'Archive metadata updated for all files.' return liveImport() } var total = importState.count.bytes var totalBar = bar({ total: total, style: function (a, b) { return `[${a}${b}] ${(100 * importState.importedBytes / total).toFixed(0)}%` } }) return output(` Creating metadata for ${importState.count.files} files ${indexSpeed} ${totalBar(importState.importedBytes)} ${fileImport(importState.fileImport)} `) function liveImport () { // Live import var imports = importState.liveImports.slice(1).slice(-7) return output(` Watching for file updates ${imports.reverse().map(function (file) { return fileImport(file) }).join('\n')} `) } function fileImport (file) { if (!file) return '' if (file.type === 'del') return `DEL: ${file.src.name}` var total = file.src.stat.size var name = file.dst.name.substr(1) // remove '/' at start var size // >500 mb show progress if (total < 5e8 || !file.progress) size = `(${pretty(total)})` else size = `(${pretty(file.progress)} / ${pretty(total)})` return output(` ADD: ${cliTruncate(name, process.stdout.columns - 7 - size.length, { position: 'start' })} ${size} `) } } ================================================ FILE: src/ui/components/network.js ================================================ var output = require('neat-log/output') var pretty = require('prettier-bytes') var pluralize = require('../elements/pluralize') module.exports = networkUI function networkUI (state) { var network = state.network var stats = state.stats if (!network) return '' var peers = stats.peers.total || 0 // var complete = stats.peers.complete return output(` ${peers} ${pluralize('connection', peers)} ${speedUI()} `) function speedUI () { var output = '| ' var speed = state.stats.network var upSpeed = speed.uploadSpeed || 0 var downSpeed = speed.downloadSpeed || 0 output += `Download ${pretty(downSpeed)}/s` output += ` Upload ${pretty(upSpeed)}/s ` return output } } ================================================ FILE: src/ui/components/sources.js ================================================ var output = require('neat-log/output') var pretty = require('prettier-bytes') var makeBar = require('progress-string') module.exports = peersUI function peersUI (state) { if (!state.network) return '' if (Object.keys(state.sources).length === 0) return '' var peers = state.sources // var stats = state.stats // var peerCount = stats.peers.total || 0 // var complete = stats.peers.complete var info = Object.keys(peers).map(function (id, i) { return peerUI(peers[id], i) }).join('\n') return `\n${info}\n` function peerUI (peer, i) { var progress = peer.getProgress() var bar = makeBar({ total: 100, style: function (a, b) { return `[${a}${b}] ${(progress).toFixed(2)}%` } }) var theBar = progress ? bar(progress) : '' // progress bar todo return output(` [${i}] ${peer.closed ? 'CLOSED' : peer.type}: ${peer.host}:${peer.port} ${pretty(peer.speed)}/s ${peer.error ? peer.error : theBar} `) } } ================================================ FILE: src/ui/components/warnings.js ================================================ var chalk = require('chalk') module.exports = function (state) { var warning = '' state.warnings.forEach(function (message) { warning += `${chalk.yellow(`Warning: ${message}`)}\n` }) return warning } ================================================ FILE: src/ui/create.js ================================================ var output = require('neat-log/output') var pretty = require('prettier-bytes') var chalk = require('chalk') var importUI = require('./components/import-progress') var keyEl = require('./elements/key') var pluralize = require('./elements/pluralize') module.exports = createUI function createUI (state) { if (!state.dat) { return output(` Creating a Dat! Add information to your dat.json file: `) } var dat = state.dat var stats = dat.stats.get() var title = '\n' var progressView var exitMsg = ` Your dat is created! Run ${chalk.green('dat sync')} to share: ${keyEl(dat.key)} ` if (!state.opts.import) { // set exiting right away state.exiting = true } if (!state.exiting) { // Only show key if not about to exit title = `${keyEl(dat.key)}\n` } if (state.title) title += state.title if (stats.version > 0) title += `: ${stats.files} ${pluralize('file', stats.files)} (${pretty(stats.byteLength)})` else if (stats.version === 0) title += ': (empty archive)' if (state.opts.import) { progressView = importUI(state) + '\n' } else { progressView = 'Not importing files.' } return output(` ${title} ${progressView} ${state.exiting ? exitMsg : chalk.dim('Ctrl+C to Exit')} `) } ================================================ FILE: src/ui/elements/key.js ================================================ var stringKey = require('dat-encoding').toStr var chalk = require('chalk') module.exports = function (key) { return `${chalk.blue(`dat://${stringKey(key)}`)}` } ================================================ FILE: src/ui/elements/pluralize.js ================================================ module.exports = function pluralize (str, val) { return `${str}${val === 1 ? '' : 's'}` } ================================================ FILE: src/ui/elements/version.js ================================================ var chalk = require('chalk') module.exports = function (version) { return `${chalk.green(`dat v${version}`)}` } ================================================ FILE: src/ui/status.js ================================================ var output = require('neat-log/output') var stringKey = require('dat-encoding').toStr var pretty = require('prettier-bytes') var chalk = require('chalk') module.exports = statusUI function statusUI (state) { if (!state.dat) return 'Starting Dat program...' var dat = state.dat var stats = dat.stats.get() return output(` ${chalk.blue('dat://' + stringKey(dat.key))} ${stats.files} files (${pretty(stats.byteLength)}) Version: ${chalk.bold(stats.version)} `) } ================================================ FILE: src/usage.js ================================================ module.exports = function (opts, help, usage) { if (opts.version) { var pkg = require('../package.json') console.error(pkg.version) process.exit(1) } var msg = ` dat [] clone or sync link to dat create and sync dat in directory Other commands: dat create create empty dat and dat.json dat sync live sync files with the network dat clone [] download a dat via link to dat pull update dat & exit dat log log history for a dat dat status get key & info about a local dat dat keys [import/export] import and export private keys Troubleshooting & Help: dat help print this usage guide dat --help, -h print help for a specific command dat --version, -v print the dat version ` console.error(msg) if (usage) { console.error('General Options:') console.error(usage) } console.error('Have fun using Dat! Learn more at docs.datproject.org') process.exit(0) } ================================================ FILE: test/auth.js ================================================ var test = require('tape') var path = require('path') var fs = require('fs') var rimraf = require('rimraf') var mkdirp = require('mkdirp') var spawn = require('./helpers/spawn') var help = require('./helpers') var authServer = require('./helpers/auth-server') var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js')) var baseTestDir = help.testFolder() var fixtures = path.join(__dirname, 'fixtures') var port = process.env.PORT || 3000 var SERVER = 'http://localhost:' + port var config = path.join(__dirname, '.datrc-test') var opts = ' --server=' + SERVER + ' --config=' + config dat += opts rimraf.sync(config) authServer(port, function (err, server, closeServer) { if (err) throw err if (!server) return test('auth - whoami works when not logged in', function (t) { var cmd = dat + ' whoami ' var st = spawn(t, cmd, { cwd: baseTestDir }) st.stderr.match(function (output) { t.same(output.trim(), 'Not logged in.', 'printed correct output') return true }) st.stdout.empty() st.end() }) test('auth - register works', function (t) { var cmd = dat + ' register --email=hello@bob.com --password=joe --username=joe' var st = spawn(t, cmd, { cwd: baseTestDir }) st.stdout.match(function (output) { t.same(output.trim(), 'Registered successfully.', 'output success message') return true }) st.stderr.empty() st.end() }) test('auth - login works', function (t) { var cmd = dat + ' login --email=hello@bob.com --password=joe' var st = spawn(t, cmd, { cwd: baseTestDir }) st.stdout.match(function (output) { t.same(output.trim(), 'Logged in successfully.', 'output success message') return true }) st.stderr.empty() st.end() }) test('auth - whoami works', function (t) { var cmd = dat + ' whoami' var st = spawn(t, cmd, { cwd: baseTestDir }) st.stdout.match(function (output) { t.same('hello@bob.com', output.trim(), 'email printed') return true }) st.stderr.empty() st.end() }) test('auth - publish before create fails', function (t) { var cmd = dat + ' publish' rimraf.sync(path.join(fixtures, '.dat')) var st = spawn(t, cmd, { cwd: fixtures }) st.stdout.empty() st.stderr.match(function (output) { t.ok(output.indexOf('existing') > -1, 'Create archive before pub') return true }) st.end() }) test('auth - create dat to publish', function (t) { rimraf.sync(path.join(fixtures, '.dat')) rimraf.sync(path.join(fixtures, 'dat.json')) var cmd = dat + ' create --no-import' var st = spawn(t, cmd, { cwd: fixtures }) st.stdout.match(function (output) { var link = help.matchLink(output) if (!link) return false t.ok(link, 'prints link') return true }) st.stderr.empty() st.end() }) test('auth - publish our awesome dat', function (t) { var cmd = dat + ' publish --name awesome' var st = spawn(t, cmd, { cwd: fixtures }) st.stdout.match(function (output) { var published = output.indexOf('Successfully published') > -1 if (!published) return false t.ok(published, 'published') return true }) st.stderr.empty() st.end() }) test('auth - publish our awesome dat with bad dat.json url', function (t) { fs.readFile(path.join(fixtures, 'dat.json'), function (err, contents) { t.ifError(err) var info = JSON.parse(contents) var oldUrl = info.url info.url = info.url.replace('e', 'a') fs.writeFile(path.join(fixtures, 'dat.json'), JSON.stringify(info), function (err) { t.ifError(err, 'error after write') var cmd = dat + ' publish --name awesome' var st = spawn(t, cmd, { cwd: fixtures }) st.stdout.match(function (output) { var published = output.indexOf('Successfully published') > -1 if (!published) return false t.ok(published, 'published') t.same(help.datJson(fixtures).url, oldUrl, 'has dat.json with url') return true }) st.stderr.empty() st.end() }) }) }) test('auth - clone from registry', function (t) { // MAKE SURE THESE MATCH WHAT is published above // TODO: be less lazy and make a publish helper var shortName = 'localhost:' + port + '/joe/awesome' // they'll never guess who wrote these tests var baseDir = path.join(baseTestDir, 'dat_registry_dir') mkdirp.sync(baseDir) var downloadDir = path.join(baseDir, shortName.split('/').pop()) var cmd = dat + ' clone ' + shortName var st = spawn(t, cmd, { cwd: baseDir }) st.stdout.match(function (output) { var lookingFor = output.indexOf('Looking for') > -1 if (!lookingFor) return false t.ok(lookingFor, 'starts looking for peers') t.ok(output.indexOf(downloadDir) > -1, 'prints dir') st.kill() return true }) st.stderr.empty() st.end(function () { rimraf.sync(downloadDir) }) }) test('auth - publish our awesome dat without a dat.json file', function (t) { rimraf(path.join(fixtures, 'dat.json'), function (err) { t.ifError(err) var cmd = dat + ' publish --name another-awesome' var st = spawn(t, cmd, { cwd: fixtures }) st.stdout.match(function (output) { var published = output.indexOf('Successfully published') > -1 if (!published) return false t.ok(published, 'published') t.same(help.datJson(fixtures).name, 'another-awesome', 'has dat.json with name') return true }) st.stderr.empty() st.end(function () { rimraf.sync(path.join(fixtures, '.dat')) }) }) }) test('auth - bad clone from registry', function (t) { var shortName = 'localhost:' + port + '/joe/not-at-all-awesome' var baseDir = path.join(baseTestDir, 'dat_registry_dir_too') mkdirp.sync(baseDir) var downloadDir = path.join(baseDir, shortName.split('/').pop()) var cmd = dat + ' clone ' + shortName var st = spawn(t, cmd, { cwd: baseDir }) st.stderr.match(function (output) { t.same(output.trim(), 'Dat with that name not found.', 'not found') st.kill() return true }) st.stdout.empty() st.end(function () { rimraf.sync(downloadDir) }) }) test('auth - logout works', function (t) { var cmd = dat + ' logout' var st = spawn(t, cmd, { cwd: baseTestDir }) st.stdout.match(function (output) { t.same('Logged out.', output.trim(), 'output correct') return true }) st.stderr.empty() st.end() }) test('auth - logout prints correctly when trying to log out twice', function (t) { var cmd = dat + ' logout' var st = spawn(t, cmd, { cwd: baseTestDir }) st.stderr.match(function (output) { t.same('Not logged in.', output.trim(), 'output correct') return true }) st.stdout.empty() st.end() }) test('auth - whoami works after logging out', function (t) { var cmd = dat + ' whoami' var st = spawn(t, cmd, { cwd: baseTestDir }) st.stderr.match(function (output) { t.same('Not logged in.', output.trim()) return true }) st.stdout.empty() st.end() }) test.onFinish(function () { closeServer(function () { fs.unlink(config, function () { // done! }) }) }) }) ================================================ FILE: test/clone.js ================================================ var fs = require('fs') var path = require('path') var test = require('tape') var tempDir = require('temporary-directory') var spawn = require('./helpers/spawn.js') var help = require('./helpers') var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js')) test('clone - default opts', function (t) { help.shareFixtures(function (_, shareDat) { var key = shareDat.key.toString('hex') tempDir(function (_, dir, cleanup) { var cmd = dat + ' clone ' + key var st = spawn(t, cmd, { cwd: dir }) var datDir = path.join(dir, key) st.stdout.match(function (output) { var downloadFinished = output.indexOf('Exiting') > -1 if (!downloadFinished) return false var stats = shareDat.stats.get() var fileRe = new RegExp(stats.files + ' files') var bytesRe = new RegExp(/1\.\d KB/) t.ok(output.match(fileRe), 'total size: files okay') t.ok(output.match(bytesRe), 'total size: bytes okay') t.ok(help.isDir(datDir), 'creates download directory') var fileList = help.fileList(datDir).join(' ') var hasCsvFile = fileList.indexOf('all_hour.csv') > -1 t.ok(hasCsvFile, 'csv file downloaded') var hasDatFolder = fileList.indexOf('.dat') > -1 t.ok(hasDatFolder, '.dat folder created') var hasSubDir = fileList.indexOf('folder') > -1 t.ok(hasSubDir, 'folder created') var hasNestedDir = fileList.indexOf('nested') > -1 t.ok(hasNestedDir, 'nested folder created') var hasHelloFile = fileList.indexOf('hello.txt') > -1 t.ok(hasHelloFile, 'hello.txt file downloaded') st.kill() return true }) st.succeeds('exits after finishing download') st.stderr.empty() st.end(function () { cleanup() shareDat.close() }) }) }) }) // Right now we aren't forcing this // test('clone - errors on existing dir', function (t) { // tempDir(function (_, dir, cleanup) { // // make empty dat in directory // Dat(dir, function (err, shareDat) { // t.error(err, 'no error') // // Try to clone to same dir // shareDat.close(function () { // var cmd = dat + ' clone ' + shareDat.key.toString('hex') + ' ' + dir // var st = spawn(t, cmd) // st.stdout.empty() // st.stderr.match(function (output) { // t.same(output.trim(), 'Existing archive in this directory. Use pull or sync to update.', 'Existing archive.') // st.kill() // return true // }) // st.end(cleanup) // }) // }) // }) // }) test('clone - specify dir', function (t) { help.shareFixtures(function (_, shareDat) { tempDir(function (_, dir, cleanup) { var key = shareDat.key.toString('hex') var customDir = 'my_dir' var cmd = dat + ' clone ' + key + ' ' + customDir var st = spawn(t, cmd, { cwd: dir }) st.stdout.match(function (output) { var downloadFinished = output.indexOf('Exiting') > -1 if (!downloadFinished) return false t.ok(help.isDir(path.join(dir, customDir)), 'creates download directory') st.kill() return true }) st.succeeds('exits after finishing download') st.stderr.empty() st.end(function () { cleanup() shareDat.close() }) }) }) }) test('clone - dat:// link', function (t) { help.shareFixtures(function (_, shareDat) { tempDir(function (_, dir, cleanup) { var key = 'dat://' + shareDat.key.toString('hex') + '/' var cmd = dat + ' clone ' + key + ' ' var downloadDir = path.join(dir, shareDat.key.toString('hex')) var st = spawn(t, cmd, { cwd: dir }) st.stdout.match(function (output) { var downloadFinished = output.indexOf('Exiting') > -1 if (!downloadFinished) return false t.ok(help.isDir(path.join(downloadDir)), 'creates download directory') st.kill() return true }) st.succeeds('exits after finishing download') st.stderr.empty() st.end(function () { cleanup() shareDat.close() }) }) }) }) test('clone - datproject.org/key link', function (t) { help.shareFixtures(function (_, shareDat) { tempDir(function (_, dir, cleanup) { var key = 'datproject.org/' + shareDat.key.toString('hex') + '/' var cmd = dat + ' clone ' + key + ' ' var downloadDir = path.join(dir, shareDat.key.toString('hex')) var st = spawn(t, cmd, { cwd: dir }) st.stdout.match(function (output) { var downloadFinished = output.indexOf('Exiting') > -1 if (!downloadFinished) return false t.ok(help.isDir(path.join(downloadDir)), 'creates download directory') st.kill() return true }) st.succeeds('exits after finishing download') st.stderr.empty() st.end(function () { cleanup() shareDat.close() }) }) }) }) // TODO: fix --temp for clones // test('clone - with --temp', function (t) { // // cmd: dat clone // help.shareFixtures(function (_, fixturesDat) { // shareDat = fixturesDat // var key = shareDat.key.toString('hex') // var cmd = dat + ' clone ' + key + ' --temp' // var st = spawn(t, cmd, {cwd: baseTestDir}) // var datDir = path.join(baseTestDir, key) // st.stdout.match(function (output) { // var downloadFinished = output.indexOf('Download Finished') > -1 // if (!downloadFinished) return false // var stats = shareDat.stats.get() // var fileRe = new RegExp(stats.filesTotal + ' files') // var bytesRe = new RegExp(/1\.\d{1,2} kB/) // t.ok(help.matchLink(output), 'prints link') // t.ok(output.indexOf('dat-download-folder/' + key) > -1, 'prints dir') // t.ok(output.match(fileRe), 'total size: files okay') // t.ok(output.match(bytesRe), 'total size: bytes okay') // t.ok(help.isDir(datDir), 'creates download directory') // var fileList = help.fileList(datDir).join(' ') // var hasCsvFile = fileList.indexOf('all_hour.csv') > -1 // t.ok(hasCsvFile, 'csv file downloaded') // var hasDatFolder = fileList.indexOf('.dat') > -1 // t.ok(!hasDatFolder, '.dat folder not created') // var hasSubDir = fileList.indexOf('folder') > -1 // t.ok(hasSubDir, 'folder created') // var hasNestedDir = fileList.indexOf('nested') > -1 // t.ok(hasNestedDir, 'nested folder created') // var hasHelloFile = fileList.indexOf('hello.txt') > -1 // t.ok(hasHelloFile, 'hello.txt file downloaded') // st.kill() // return true // }) // st.succeeds('exits after finishing download') // st.stderr.empty() // st.end() // }) // }) test('clone - invalid link', function (t) { var key = 'best-key-ever' var cmd = dat + ' clone ' + key tempDir(function (_, dir, cleanup) { var st = spawn(t, cmd, { cwd: dir }) var datDir = path.join(dir, key) st.stderr.match(function (output) { var error = output.indexOf('Could not resolve link') > -1 if (!error) return false t.ok(error, 'has error') t.ok(!help.isDir(datDir), 'download dir removed') st.kill() return true }) st.end(cleanup) }) }) test('clone - shortcut/stateless clone', function (t) { help.shareFixtures(function (_, shareDat) { var key = shareDat.key.toString('hex') tempDir(function (_, dir, cleanup) { var datDir = path.join(dir, key) var cmd = dat + ' ' + key + ' ' + datDir + ' --exit' var st = spawn(t, cmd) st.stdout.match(function (output) { var downloadFinished = output.indexOf('Exiting') > -1 if (!downloadFinished) return false t.ok(help.isDir(datDir), 'creates download directory') var fileList = help.fileList(datDir).join(' ') var hasCsvFile = fileList.indexOf('all_hour.csv') > -1 t.ok(hasCsvFile, 'csv file downloaded') var hasDatFolder = fileList.indexOf('.dat') > -1 t.ok(hasDatFolder, '.dat folder created') var hasSubDir = fileList.indexOf('folder') > -1 t.ok(hasSubDir, 'folder created') var hasNestedDir = fileList.indexOf('nested') > -1 t.ok(hasNestedDir, 'nested folder created') var hasHelloFile = fileList.indexOf('hello.txt') > -1 t.ok(hasHelloFile, 'hello.txt file downloaded') st.kill() return true }) st.succeeds('exits after finishing download') st.stderr.empty() st.end(function () { cleanup() shareDat.close() }) }) }) }) // TODO: fix this // test('clone - hypercore link', function (t) { // help.shareFeed(function (_, key, close) { // tempDir(function (_, dir, cleanup) { // var cmd = dat + ' clone ' + key // var st = spawn(t, cmd, {cwd: dir}) // var datDir = path.join(dir, key) // st.stderr.match(function (output) { // var error = output.indexOf('not a Dat Archive') > -1 // if (!error) return false // t.ok(error, 'has error') // t.ok(!help.isDir(datDir), 'download dir removed') // st.kill() // return true // }) // st.end(function () { // cleanup() // close() // }) // }) // }) // }) test('clone - specify directory containing dat.json', function (t) { help.shareFixtures(function (_, shareDat) { tempDir(function (_, dir, cleanup) { fs.writeFileSync(path.join(dir, 'dat.json'), JSON.stringify({ url: shareDat.key.toString('hex') }), 'utf8') // dat clone /dir var cmd = dat + ' clone ' + dir var st = spawn(t, cmd) var datDir = dir st.stdout.match(function (output) { var downloadFinished = output.indexOf('Exiting') > -1 if (!downloadFinished) return false var fileList = help.fileList(datDir).join(' ') var hasCsvFile = fileList.indexOf('all_hour.csv') > -1 t.ok(hasCsvFile, 'csv file downloaded') var hasDatFolder = fileList.indexOf('.dat') > -1 t.ok(hasDatFolder, '.dat folder created') var hasSubDir = fileList.indexOf('folder') > -1 t.ok(hasSubDir, 'folder created') var hasNestedDir = fileList.indexOf('nested') > -1 t.ok(hasNestedDir, 'nested folder created') var hasHelloFile = fileList.indexOf('hello.txt') > -1 t.ok(hasHelloFile, 'hello.txt file downloaded') st.kill() return true }) st.succeeds('exits after finishing download') st.stderr.empty() st.end(function () { cleanup() shareDat.close() }) }) }) }) test('clone - specify directory containing dat.json with cwd', function (t) { help.shareFixtures(function (_, shareDat) { tempDir(function (_, dir, cleanup) { fs.writeFileSync(path.join(dir, 'dat.json'), JSON.stringify({ url: shareDat.key.toString('hex') }), 'utf8') // cd dir && dat clone /dir/dat.json var cmd = dat + ' clone ' + dir var st = spawn(t, cmd, { cwd: dir }) var datDir = dir st.stdout.match(function (output) { var downloadFinished = output.indexOf('Exiting') > -1 if (!downloadFinished) return false var fileList = help.fileList(datDir).join(' ') var hasCsvFile = fileList.indexOf('all_hour.csv') > -1 t.ok(hasCsvFile, 'csv file downloaded') var hasDatFolder = fileList.indexOf('.dat') > -1 t.ok(hasDatFolder, '.dat folder created') var hasSubDir = fileList.indexOf('folder') > -1 t.ok(hasSubDir, 'folder created') var hasNestedDir = fileList.indexOf('nested') > -1 t.ok(hasNestedDir, 'nested folder created') var hasHelloFile = fileList.indexOf('hello.txt') > -1 t.ok(hasHelloFile, 'hello.txt file downloaded') st.kill() return true }) st.succeeds('exits after finishing download') st.stderr.empty() st.end(function () { cleanup() shareDat.close() }) }) }) }) test('clone - specify dat.json path', function (t) { help.shareFixtures(function (_, shareDat) { tempDir(function (_, dir, cleanup) { var datJsonPath = path.join(dir, 'dat.json') fs.writeFileSync(datJsonPath, JSON.stringify({ url: shareDat.key.toString('hex') }), 'utf8') // dat clone /dir/dat.json var cmd = dat + ' clone ' + datJsonPath var st = spawn(t, cmd) var datDir = dir st.stdout.match(function (output) { var downloadFinished = output.indexOf('Exiting') > -1 if (!downloadFinished) return false var fileList = help.fileList(datDir).join(' ') var hasCsvFile = fileList.indexOf('all_hour.csv') > -1 t.ok(hasCsvFile, 'csv file downloaded') var hasDatFolder = fileList.indexOf('.dat') > -1 t.ok(hasDatFolder, '.dat folder created') var hasSubDir = fileList.indexOf('folder') > -1 t.ok(hasSubDir, 'folder created') var hasNestedDir = fileList.indexOf('nested') > -1 t.ok(hasNestedDir, 'nested folder created') var hasHelloFile = fileList.indexOf('hello.txt') > -1 t.ok(hasHelloFile, 'hello.txt file downloaded') st.kill() return true }) st.succeeds('exits after finishing download') st.stderr.empty() st.end(function () { cleanup() shareDat.close() }) }) }) }) test('clone - specify dat.json path with cwd', function (t) { help.shareFixtures(function (_, shareDat) { tempDir(function (_, dir, cleanup) { var datJsonPath = path.join(dir, 'dat.json') fs.writeFileSync(datJsonPath, JSON.stringify({ url: shareDat.key.toString('hex') }), 'utf8') // cd /dir && dat clone /dir/dat.json var cmd = dat + ' clone ' + datJsonPath var st = spawn(t, cmd, { cwd: dir }) var datDir = dir st.stdout.match(function (output) { var downloadFinished = output.indexOf('Exiting') > -1 if (!downloadFinished) return false var fileList = help.fileList(datDir).join(' ') var hasCsvFile = fileList.indexOf('all_hour.csv') > -1 t.ok(hasCsvFile, 'csv file downloaded') var hasDatFolder = fileList.indexOf('.dat') > -1 t.ok(hasDatFolder, '.dat folder created') var hasSubDir = fileList.indexOf('folder') > -1 t.ok(hasSubDir, 'folder created') var hasNestedDir = fileList.indexOf('nested') > -1 t.ok(hasNestedDir, 'nested folder created') var hasHelloFile = fileList.indexOf('hello.txt') > -1 t.ok(hasHelloFile, 'hello.txt file downloaded') st.kill() return true }) st.succeeds('exits after finishing download') st.stderr.empty() st.end(function () { cleanup() shareDat.close() }) }) }) }) test('clone - specify dat.json + directory', function (t) { help.shareFixtures(function (_, shareDat) { tempDir(function (_, dir, cleanup) { var datDir = path.join(dir, 'clone-dest') var datJsonPath = path.join(dir, 'dat.json') // make dat.json in different dir fs.mkdirSync(datDir) fs.writeFileSync(datJsonPath, JSON.stringify({ url: shareDat.key.toString('hex') }), 'utf8') // dat clone /dir/dat.json /dir/clone-dest var cmd = dat + ' clone ' + datJsonPath + ' ' + datDir var st = spawn(t, cmd) st.stdout.match(function (output) { var downloadFinished = output.indexOf('Exiting') > -1 if (!downloadFinished) return false var fileList = help.fileList(datDir).join(' ') var hasCsvFile = fileList.indexOf('all_hour.csv') > -1 t.ok(hasCsvFile, 'csv file downloaded') var hasDatFolder = fileList.indexOf('.dat') > -1 t.ok(hasDatFolder, '.dat folder created') var hasSubDir = fileList.indexOf('folder') > -1 t.ok(hasSubDir, 'folder created') var hasNestedDir = fileList.indexOf('nested') > -1 t.ok(hasNestedDir, 'nested folder created') var hasHelloFile = fileList.indexOf('hello.txt') > -1 t.ok(hasHelloFile, 'hello.txt file downloaded') st.kill() return true }) st.succeeds('exits after finishing download') st.stderr.empty() st.end(function () { cleanup() shareDat.close() }) }) }) }) ================================================ FILE: test/create.js ================================================ var fs = require('fs') var path = require('path') var test = require('tape') var tempDir = require('temporary-directory') var Dat = require('dat-node') var spawn = require('./helpers/spawn.js') var help = require('./helpers') var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js')) var fixtures = path.join(__dirname, 'fixtures') // os x adds this if you view the fixtures in finder and breaks the file count assertions try { fs.unlinkSync(path.join(fixtures, '.DS_Store')) } catch (e) { /* ignore error */ } // start without dat.json try { fs.unlinkSync(path.join(fixtures, 'dat.json')) } catch (e) { /* ignore error */ } test('create - default opts no import', function (t) { tempDir(function (_, dir, cleanup) { var cmd = dat + ' create --title data --description thing' var st = spawn(t, cmd, { cwd: dir }) st.stdout.match(function (output) { var datCreated = output.indexOf('Created empty Dat') > -1 if (!datCreated) return false t.ok(help.isDir(path.join(dir, '.dat')), 'creates dat directory') st.kill() return true }) st.succeeds('exits after create finishes') st.stderr.empty() st.end(cleanup) }) }) test('create - errors on existing archive', function (t) { tempDir(function (_, dir, cleanup) { Dat(dir, function (err, dat) { t.error(err, 'no error') dat.close(function () { var cmd = dat + ' create --title data --description thing' var st = spawn(t, cmd, { cwd: dir }) st.stderr.match(function (output) { t.ok(output, 'errors') st.kill() return true }) st.end() }) }) }) }) test('create - sync after create ok', function (t) { tempDir(function (_, dir, cleanup) { var cmd = dat + ' create --title data --description thing' var st = spawn(t, cmd, { cwd: dir, end: false }) st.stdout.match(function (output) { var connected = output.indexOf('Created empty Dat') > -1 if (!connected) return false doSync() return true }) function doSync () { var cmd = dat + ' sync ' var st = spawn(t, cmd, { cwd: dir }) st.stdout.match(function (output) { var connected = output.indexOf('Sharing') > -1 if (!connected) return false st.kill() return true }) st.stderr.empty() st.end(cleanup) } }) }) test('create - init alias', function (t) { tempDir(function (_, dir, cleanup) { var cmd = dat + ' init --title data --description thing' var st = spawn(t, cmd, { cwd: dir }) st.stdout.match(function (output) { var datCreated = output.indexOf('Created empty Dat') > -1 if (!datCreated) return false t.ok(help.isDir(path.join(dir, '.dat')), 'creates dat directory') st.kill() return true }) st.succeeds('exits after create finishes') st.stderr.empty() st.end(cleanup) }) }) test('create - with path', function (t) { tempDir(function (_, dir, cleanup) { var cmd = dat + ' init ' + dir + ' --title data --description thing' var st = spawn(t, cmd) st.stdout.match(function (output) { var datCreated = output.indexOf('Created empty Dat') > -1 if (!datCreated) return false t.ok(help.isDir(path.join(dir, '.dat')), 'creates dat directory') st.kill() return true }) st.succeeds('exits after create finishes') st.stderr.empty() st.end(cleanup) }) }) ================================================ FILE: test/dat-node.js ================================================ var test = require('tape') var ram = require('random-access-memory') var Dat = require('..') test('dat-node: require dat-node + make a dat', function (t) { Dat(ram, function (err, dat) { t.error(err, 'no error') t.ok(dat, 'makes dat') t.pass('yay') t.end() }) }) ================================================ FILE: test/doctor.js ================================================ // var path = require('path') // var test = require('tape') // var spawn = require('./helpers/spawn.js') // var help = require('./helpers') // // TODO // dat-doctor requires interactive testing right now... // var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js')) // test('misc - doctor option works ', function (t) { // var st = spawn(t, dat + ' doctor', {end: false}) // st.stderr.match(function (output) { // var readyPeer = output.indexOf('Waiting for incoming connections') > -1 // if (!readyPeer) return false // if (!process.env.TRAVIS) { // // Not working on v4/v7 travis but can't reproduce locally // t.ok(output.indexOf('UTP') > -1, 'doctor connects to public peer via UTP') // } // t.ok(output.indexOf('TCP') > -1, 'doctor connects to public peer via TCP') // var key = help.matchLink(output) // startPhysiciansAssistant(key) // return true // }, 'doctor started') // function startPhysiciansAssistant (link) { // var assist = spawn(t, dat + ' doctor ' + link, {end: false}) // assist.stderr.match(function (output) { // var readyPeer = output.indexOf('Waiting for incoming connections') > -1 // if (!readyPeer) return false // t.same(help.matchLink(output), link, 'key of peer matches') // t.ok(readyPeer, 'starts looking for peers') // t.skip(output.indexOf('Remote peer echoed expected data back') > -1, 'echo data back') // st.kill() // return true // }) // assist.end(function () { // t.end() // }) // } // }) ================================================ FILE: test/fixtures/all_hour.csv ================================================ time,latitude,longitude,depth,mag,magType,nst,gap,dmin,rms,net,id,updated,place,type 2014-04-30T03:34:57.000Z,60.0366,-141.2214,14.6,1.4,ml,,,,1.51,ak,ak11246293,2014-04-30T03:39:27.956Z,"67km E of Cape Yakataga, Alaska",earthquake 2014-04-30T03:16:54.860Z,33.9233322,-117.9376678,0.81,2.4,ml,71,51,0.02487,0.35,ci,ci37218696,2014-04-30T03:35:24.239Z,"1km SE of La Habra, California",earthquake 2014-04-30T03:03:09.000Z,61.126,-149.7035,28.2,1,ml,,,,0.75,ak,ak11246291,2014-04-30T03:09:51.716Z,"14km SE of Anchorage, Alaska",earthquake 2014-04-30T02:57:51.800Z,37.3798,-122.1912,4.5,1.3,Md,8,104.4,0.01796631,0.02,nc,nc72212216,2014-04-30T03:28:05.271Z,"2km SSE of Ladera, California",earthquake 2014-04-30T02:51:26.000Z,60.2062,-147.7298,0,1.8,ml,,,,0.1,ak,ak11246289,2014-04-30T02:54:55.916Z,"82km SE of Whittier, Alaska",earthquake 2014-04-30T02:48:54.000Z,60.4899,-149.4879,28.5,1.4,ml,,,,0.54,ak,ak11246287,2014-04-30T02:54:51.149Z,"36km N of Bear Creek, Alaska",earthquake 2014-04-30T02:47:44.100Z,38.819,-122.7952,3.3,0.8,Md,12,75.6,0.00898315,0.02,nc,nc72212211,2014-04-30T03:17:09.173Z,"5km NW of The Geysers, California",earthquake 2014-04-30T02:46:11.100Z,37.9812,-122.054,14.7,2.1,Md,35,126,0.09881468,0.09,nc,nc72212206,2014-04-30T03:38:06.391Z,"1km E of Pacheco, California",earthquake 2014-04-30T02:44:49.000Z,61.3482,-151.3611,48.5,1.7,ml,,,,0.92,ak,ak11246285,2014-04-30T02:54:49.809Z,"73km N of Nikiski, Alaska",earthquake ================================================ FILE: test/fixtures/folder/nested/hello.txt ================================================ code for science and society ================================================ FILE: test/helpers/auth-server.js ================================================ var path = require('path') var rimraf = require('rimraf') var Server, initDb try { Server = require('dat-registry-api/server') initDb = require('dat-registry-api/server/database/init') } catch (e) { console.log('Disabling auth tests, run `npm install dat-registry-api` to enable them.') } module.exports = createServer function createServer (port, cb) { if (!Server || !initDb) return cb(null) var config = { mixpanel: 'nothing', email: { fromEmail: 'hi@example.com' }, township: { secret: 'very secret code', db: path.join(__dirname, '..', 'test-township.db') }, db: { dialect: 'sqlite3', connection: { filename: path.join(__dirname, '..', 'test-sqlite.db') }, useNullAsDefault: true }, archiver: path.join(__dirname, '..', 'test-archiver'), whitelist: false, port: port || 8888 } rimraf.sync(config.archiver) rimraf.sync(config.db.connection.filename) rimraf.sync(config.township.db) initDb(config.db, function (err, db) { if (err) return cb(err) const server = Server(config, db) server.listen(config.port, function () { console.log('listening', config.port) }) cb(null, server, close) function close (cb) { server.close(function () { rimraf.sync(config.township.db) rimraf.sync(config.db.connection.filename) process.exit() }) } }) } ================================================ FILE: test/helpers/index.js ================================================ var fs = require('fs') var os = require('os') var path = require('path') var mkdirp = require('mkdirp') var rimraf = require('rimraf') var encoding = require('dat-encoding') var recursiveReadSync = require('recursive-readdir-sync') var Dat = require('dat-node') var fetch = require('node-fetch') module.exports.matchLink = matchDatLink module.exports.isDir = isDir module.exports.testFolder = newTestFolder module.exports.datJson = datJson module.exports.shareFixtures = shareFixtures module.exports.fileList = fileList module.exports.fetchText = fetchText function shareFixtures (opts, cb) { if (typeof opts === 'function') return shareFixtures(null, opts) opts = opts || {} var fixtures = path.join(__dirname, '..', 'fixtures') // os x adds this if you view the fixtures in finder and breaks the file count assertions try { fs.unlinkSync(path.join(fixtures, '.DS_Store')) } catch (e) { /* ignore error */ } if (opts.resume !== true) rimraf.sync(path.join(fixtures, '.dat')) Dat(fixtures, {}, function (err, dat) { if (err) throw err dat.trackStats() dat.joinNetwork() if (opts.import === false) return cb(null, dat) dat.importFiles({ watch: false }, function (err) { if (err) throw err cb(null, dat) }) }) } function fileList (dir) { try { return recursiveReadSync(dir) } catch (e) { return [] } } function newTestFolder () { var tmpdir = path.join(os.tmpdir(), 'dat-download-folder') rimraf.sync(tmpdir) mkdirp.sync(tmpdir) return tmpdir } function matchDatLink (str) { var match = str.match(/[A-Za-z0-9]{64}/) if (!match) return false var key try { key = encoding.toStr(match[0].trim()) } catch (e) { return false } return key } function datJson (filepath) { try { return JSON.parse(fs.readFileSync(path.join(filepath, 'dat.json'))) } catch (e) { return {} } } function isDir (dir) { try { return fs.statSync(dir).isDirectory() } catch (e) { return false } } function fetchText (url) { let resp = {} return fetch(url).then(function (res) { resp = res return resp.text() }).then(function (body) { return { resp, body, error: null } }).catch(function (error) { return { error, resp: {}, body: null } }) } ================================================ FILE: test/helpers/spawn.js ================================================ var spawn = require('tape-spawn') var fs = require('fs') // happens once at require time // https://github.com/AndreasMadsen/execspawn/issues/2 var hasBash = fs.existsSync('/bin/bash') module.exports = function (t, cmd, opts) { opts = opts || {} if (hasBash) opts.shell = '/bin/bash' // override default of /bin/sh return spawn(t, cmd, opts) } ================================================ FILE: test/http.js ================================================ var path = require('path') var test = require('tape') var rimraf = require('rimraf') var spawn = require('./helpers/spawn.js') var { fetchText } = require('./helpers') var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js')) var fixtures = path.join(__dirname, 'fixtures') test('http - share with http', function (t) { rimraf.sync(path.join(fixtures, '.dat')) var cmd = dat + ' share --http' var st = spawn(t, cmd, { cwd: fixtures }) st.stdout.match(function (output) { var sharingHttp = output.indexOf('Serving files over http') > -1 if (!sharingHttp) return false fetchText('http://localhost:8080').then(function ({ resp, body, error }) { t.error(error, 'no error') t.ok(resp.status === 200, 'okay status') t.ok(body) fetchText('http://localhost:8080/folder/nested/hello.txt').then(function ({ resp, body, error }) { t.error(error, 'no error') t.ok(resp.status === 200, 'okay status') t.same(body, 'code for science and society', 'body of file okay') st.kill() }) }) return true }) st.stderr.empty() st.end() }) test('http - share with http other port', function (t) { rimraf.sync(path.join(fixtures, '.dat')) var cmd = dat + ' share --http 3333' var st = spawn(t, cmd, { cwd: fixtures }) st.stdout.match(function (output) { var sharingHttp = output.indexOf('Serving files over http') > -1 if (!sharingHttp) return false fetchText('http://localhost:3333').then(function ({ resp, body, error }) { t.error(error, 'no error') t.ok(resp.status === 200, 'okay status') t.ok(body) fetchText('http://localhost:3333/folder/nested/hello.txt').then(function ({ resp, body, error }) { t.error(error, 'no error') t.ok(resp.status === 200, 'okay status') t.same(body, 'code for science and society', 'body of file okay') st.kill() }) }) return true }) st.stderr.empty() st.end() }) ================================================ FILE: test/keys.js ================================================ var fs = require('fs') var path = require('path') var test = require('tape') var rimraf = require('rimraf') var tempDir = require('temporary-directory') var spawn = require('./helpers/spawn.js') var help = require('./helpers') var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js')) var fixtures = path.join(__dirname, 'fixtures') test('keys - print keys', function (t) { help.shareFixtures(function (_, shareDat) { shareDat.close(function () { var cmd = dat + ' keys ' var st = spawn(t, cmd, { cwd: fixtures }) st.stdout.match(function (output) { if (output.indexOf('dat://') === -1) return false t.ok(output.indexOf(shareDat.key.toString('hex') > -1), 'prints key') st.kill() return true }) st.stderr.empty() st.end() }) }) }) test('keys - print discovery key', function (t) { help.shareFixtures(function (_, shareDat) { shareDat.close(function () { var cmd = dat + ' keys --discovery' var st = spawn(t, cmd, { cwd: fixtures }) st.stdout.match(function (output) { if (output.indexOf('Discovery') === -1) return false t.ok(output.indexOf(shareDat.key.toString('hex') > -1), 'prints key') t.ok(output.indexOf(shareDat.archive.discoveryKey.toString('hex') > -1), 'prints discovery key') st.kill() return true }) st.stderr.empty() st.end() }) }) }) if (!process.env.TRAVIS) { test('keys - export & import secret key', function (t) { help.shareFixtures(function (_, shareDat) { var key = shareDat.key.toString('hex') tempDir(function (_, dir, cleanup) { var cmd = dat + ' clone ' + key var st = spawn(t, cmd, { cwd: dir, end: false }) var datDir = path.join(dir, key) st.stdout.match(function (output) { var downloadFinished = output.indexOf('Exiting') > -1 if (!downloadFinished) return false st.kill() shareDat.close(exchangeKeys) return true }) st.stderr.empty() function exchangeKeys () { var secretKey = null var exportKey = dat + ' keys export' var st = spawn(t, exportKey, { cwd: fixtures, end: false }) st.stdout.match(function (output) { if (!output) return false secretKey = output.trim() st.kill() importKey() return true }) st.stderr.empty() function importKey () { var exportKey = dat + ' keys import' var st = spawn(t, exportKey, { cwd: datDir }) st.stdout.match(function (output) { if (!output.indexOf('secret key') === -1) return false st.stdin.write(secretKey + '\r') if (output.indexOf('Successful import') === -1) return false t.ok(fs.statSync(path.join(datDir, '.dat', 'metadata.ogd')), 'original dat file exists') st.kill() return true }) st.stderr.empty() st.end(function () { rimraf.sync(path.join(fixtures, '.dat')) cleanup() }) } } }) }) }) } ================================================ FILE: test/pull.js ================================================ var path = require('path') var test = require('tape') var tempDir = require('temporary-directory') var spawn = require('./helpers/spawn.js') var help = require('./helpers') var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js')) test('pull - errors without clone first', function (t) { tempDir(function (_, dir, cleanup) { var cmd = dat + ' pull' var st = spawn(t, cmd, { cwd: dir }) st.stderr.match(function (output) { t.ok('No existing archive', 'Error: no existing archive') st.kill() return true }) st.end(cleanup) }) }) test('pull - default opts', function (t) { // import false so we can pull files later help.shareFixtures({ import: false }, function (_, fixturesDat) { tempDir(function (_, dir, cleanup) { // clone initial dat var cmd = dat + ' clone ' + fixturesDat.key.toString('hex') + ' ' + dir var st = spawn(t, cmd, { end: false }) st.stdout.match(function (output) { var synced = output.indexOf('dat synced') > -1 if (!synced) return false st.kill() fixturesDat.close(doPull) return true }) function doPull () { // TODO: Finish this one. Need some bug fixes on empty pulls =( help.shareFixtures({ resume: true, import: true }, function (_, fixturesDat) { var cmd = dat + ' pull' var st = spawn(t, cmd, { cwd: dir }) st.stdout.match(function (output) { var downloadFinished = output.indexOf('dat sync') > -1 if (!downloadFinished) return false st.kill() return true }) st.succeeds('exits after finishing download') st.stderr.empty() st.end(function () { fixturesDat.close() }) }) } }) }) }) // test('pull - default opts', function (t) { // // cmd: dat pull // // import the files to the sharer so we can pull new data // shareDat.importFiles(function (err) { // if (err) throw err // var datDir = path.join(baseTestDir, shareKey) // var cmd = dat + ' pull' // var st = spawn(t, cmd, {cwd: datDir}) // st.stdout.match(function (output) { // var downloadFinished = output.indexOf('Download Finished') > -1 // if (!downloadFinished) return false // var stats = shareDat.stats.get() // var fileRe = new RegExp(stats.filesTotal + ' files') // var bytesRe = new RegExp(/1\.\d{1,2} kB/) // t.ok(help.matchLink(output), 'prints link') // t.ok(output.indexOf('dat-download-folder/' + shareKey) > -1, 'prints dir') // t.ok(output.match(fileRe), 'total size: files okay') // t.ok(output.match(bytesRe), 'total size: bytes okay') // t.ok(help.isDir(datDir), 'creates download directory') // var fileList = help.fileList(datDir).join(' ') // var hasCsvFile = fileList.indexOf('all_hour.csv') > -1 // t.ok(hasCsvFile, 'csv file downloaded') // var hasDatFolder = fileList.indexOf('.dat') > -1 // t.ok(hasDatFolder, '.dat folder created') // var hasSubDir = fileList.indexOf('folder') > -1 // t.ok(hasSubDir, 'folder created') // var hasNestedDir = fileList.indexOf('nested') > -1 // t.ok(hasNestedDir, 'nested folder created') // var hasHelloFile = fileList.indexOf('hello.txt') > -1 // t.ok(hasHelloFile, 'hello.txt file downloaded') // st.kill() // return true // }) // st.succeeds('exits after finishing download') // st.stderr.empty() // st.end() // }) // }) // test('pull - with dir arg', function (t) { // var dirName = shareKey // var datDir = path.join(baseTestDir, shareKey) // var cmd = dat + ' pull ' + dirName // var st = spawn(t, cmd, {cwd: baseTestDir}) // st.stdout.match(function (output) { // var downloadFinished = output.indexOf('Download Finished') > -1 // if (!downloadFinished) return false // t.ok(output.indexOf('dat-download-folder/' + dirName) > -1, 'prints dir') // t.ok(help.isDir(datDir), 'creates download directory') // st.kill() // return true // }) // st.succeeds('exits after finishing download') // st.stderr.empty() // st.end() // }) ================================================ FILE: test/share.js ================================================ // var fs = require('fs') // var path = require('path') // var test = require('tape') // var rimraf = require('rimraf') // var spawn = require('./helpers/spawn.js') // var help = require('./helpers') // var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js')) // if (process.env.TRAVIS) dat += ' --no-watch ' // var fixtures = path.join(__dirname, 'fixtures') // // os x adds this if you view the fixtures in finder and breaks the file count assertions // try { fs.unlinkSync(path.join(fixtures, '.DS_Store')) } catch (e) { /* ignore error */ } // // start without dat.json // try { fs.unlinkSync(path.join(fixtures, 'dat.json')) } catch (e) { /* ignore error */ } // test('share - default opts', function (t) { // rimraf.sync(path.join(fixtures, '.dat')) // var cmd = dat + ' share' // var st = spawn(t, cmd, {cwd: fixtures}) // st.stdout.match(function (output) { // var importFinished = output.indexOf('Total Size') > -1 // if (!importFinished) return false // t.ok(help.isDir(path.join(fixtures, '.dat')), 'creates dat directory') // t.ok(output.indexOf('Looking for connections') > -1, 'network') // st.kill() // return true // }) // st.stderr.empty() // st.end() // }) // test('share - with dir arg', function (t) { // rimraf.sync(path.join(fixtures, '.dat')) // var cmd = dat + ' share ' + fixtures // var st = spawn(t, cmd) // st.stdout.match(function (output) { // var importFinished = output.indexOf('Total Size') > -1 // if (!importFinished) return false // t.ok(help.isDir(path.join(fixtures, '.dat')), 'creates dat directory') // t.ok(output.indexOf('Looking for connections') > -1, 'network') // st.kill() // return true // }) // st.stderr.empty() // st.end() // }) // test.onFinish(function () { // rimraf.sync(path.join(fixtures, '.dat')) // }) ================================================ FILE: test/sync-owner.js ================================================ // var fs = require('fs') // var net = require('net') // var path = require('path') // var test = require('tape') // var mkdirp = require('mkdirp') // var rimraf = require('rimraf') // var Dat = require('dat-node') // var spawn = require('./helpers/spawn.js') // var help = require('./helpers') // var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js')) // if (process.env.TRAVIS) dat += ' --no-watch ' // var fixtures = path.join(__dirname, 'fixtures') // var downDat // // os x adds this if you view the fixtures in finder and breaks the file count assertions // try { fs.unlinkSync(path.join(fixtures, '.DS_Store')) } catch (e) { /* ignore error */ } // test('sync-owner - errors without create first', function (t) { // rimraf.sync(path.join(fixtures, '.dat')) // // cmd: dat sync // var cmd = dat + ' sync' // var st = spawn(t, cmd, {cwd: fixtures}) // st.stderr.match(function (output) { // var hasError = output.indexOf('No existing archive') > -1 // t.ok(hasError, 'emits error') // st.kill() // return true // }) // st.end() // }) // test('sync-owner - create a dat for syncing', function (t) { // rimraf.sync(path.join(fixtures, '.dat')) // // cmd: dat create // var cmd = dat + ' create --import' // var st = spawn(t, cmd, {cwd: fixtures}) // st.stdout.match(function (output) { // var importFinished = output.indexOf('import finished') > -1 // if (!importFinished) return false // st.kill() // return true // }) // st.stderr.empty() // st.end() // }) // test('sync-owner - default opts', function (t) { // // cmd: dat sync // var cmd = dat + ' sync' // var st = spawn(t, cmd, {cwd: fixtures, end: false}) // var key // st.stdout.match(function (output) { // var sharing = output.indexOf('Dat Network') > -1 // if (!sharing) return false // key = help.matchLink(output) // t.ok(key, 'prints link') // t.ok(output.indexOf('tests/fixtures') > -1, 'prints dir') // downloadDat() // return true // }) // st.stderr.empty() // st.end() // function downloadDat () { // var downloadDir = path.join(help.testFolder(), '' + Date.now()) // mkdirp.sync(downloadDir) // Dat(downloadDir, { key: key }, function (err, tmpDat) { // if (err) throw err // downDat = tmpDat // downDat.joinNetwork() // downDat.network.swarm.once('connection', function () { // t.pass('downloader connects') // downDat.close(function () { // rimraf.sync(downDat.path) // t.end() // }) // }) // }) // } // }) // test('sync-owner - create without import for syncing', function (t) { // rimraf.sync(path.join(fixtures, '.dat')) // // cmd: dat create // var cmd = dat + ' create' // var st = spawn(t, cmd, {cwd: fixtures}) // st.stdout.match(function (output) { // if (output.indexOf('created') > -1) return true // return false // }) // st.succeeds() // st.end() // }) // test('sync-owner - imports after no-import create', function (t) { // // cmd: dat sync // var cmd = dat + ' sync' // var st = spawn(t, cmd, {cwd: fixtures}) // st.stdout.match(function (output) { // // have to check both for local test (watching) and travis (sharing) // var sharing = output.indexOf('Watching') > -1 || output.indexOf('Sharing latest') > -1 // if (!sharing) return false // var fileRe = new RegExp('2 files') // var bytesRe = new RegExp(/1\.\d{1,2} kB/) // t.ok(help.matchLink(output), 'prints link') // t.ok(output.indexOf('tests/fixtures') > -1, 'prints dir') // t.ok(output.match(fileRe), 'total size: files okay') // t.ok(output.match(bytesRe), 'total size: bytes okay') // st.kill() // return true // }) // st.stderr.empty() // st.end() // }) // // TODO: this test is causing serious memory issues. // // HELP. Maybe related to https://github.com/datproject/dat-node/issues/71 // // test('sync-owner - turn off ignore hidden', function (t) { // // // cmd: dat sync // // var hiddenFile = path.join(fixtures, '.hidden-file') // // var cmd = dat + ' sync --no-ignore-hidden' // // fs.writeFile(hiddenFile, 'You cannot see me', function (err) { // // t.error(err) // // var st = spawn(t, cmd, {cwd: fixtures, end: false}) // // var key // // st.stdout.match(function (output) { // // var sharing = output.indexOf('Dat Network') > -1 // // if (!sharing) return false // // key = help.matchLink(output) // // downloadDat() // // return true // // }) // // st.stderr.empty() // // st.end() // // function downloadDat () { // // var downloadDir = path.join(help.testFolder(), '' + Date.now()) // // mkdirp.sync(downloadDir) // // Dat(downloadDir, { key: key }, function (err, downDat) { // // if (err) throw err // // downDat.joinNetwork() // // downDat.network.swarm.once('connection', function () { // // downDat.archive.list({live: false}, function (err, data) { // // t.error(err) // // var hasHiddenFile = data.filter(function (entry) { // // return entry.name === '.hidden-file' // // }) // // t.ok(hasHiddenFile.length, 'hidden file in archive') // // downDat.network.swarm.close(function () { // // process.nextTick(function () { // // downDat.close(function () { // // rimraf(downDat.path, function () { // // fs.unlink(hiddenFile, function () { // // t.end() // // }) // // }) // // }) // // }) // // }) // // }) // // }) // // }) // // } // // }) // // }) // test('sync-owner - port and utp options', function (t) { // var port = 3281 // var cmd = dat + ' sync --port ' + port + ' --no-utp' // var st = spawn(t, cmd, {cwd: fixtures, end: false}) // st.stderr.empty() // var server = net.createServer() // server.once('error', function (err) { // if (err.code !== 'EADDRINUSE') return t.error(err) // t.skip('TODO: correct port in use') // done() // }) // server.once('listening', function () { // t.skip(`TODO: port ${server.address().port} should be in use`) // done() // }) // server.listen(port) // t.skip('TODO: check utp option') // TODO: how to check utp? // function done () { // server.close(function () { // st.kill() // t.end() // }) // } // }) // test('sync-owner - shorthand', function (t) { // var cmd = dat + ' .' // var st = spawn(t, cmd, {cwd: fixtures}) // st.stdout.match(function (output) { // var sharing = output.indexOf('Looking for connections') > -1 // if (!sharing) return false // t.ok(help.matchLink(output), 'prints link') // st.kill() // return true // }) // st.stderr.empty() // st.end() // }) // test('sync-owner - dir argument', function (t) { // var cmd = dat + ' sync ' + fixtures // var st = spawn(t, cmd) // st.stdout.match(function (output) { // var sharing = output.indexOf('Looking for connections') > -1 // if (!sharing) return false // t.ok(help.matchLink(output), 'prints link') // st.kill() // return true // }) // st.stderr.empty() // st.end() // }) // if (!process.env.TRAVIS) { // test('sync-owner - live', function (t) { // var liveFile = path.join(fixtures, 'live.txt') // var wroteFile = false // var cmd = dat + ' sync --watch' // var st = spawn(t, cmd, {cwd: fixtures}) // st.stdout.match(function (output) { // var watching = output.indexOf('Watching for file changes') > -1 // if (!watching) return false // else if (!wroteFile) { // fs.writeFileSync(liveFile, 'hello') // wroteFile = true // } // var fileImported = output.indexOf('live.txt') > -1 // if (!fileImported) return false // t.ok(fileImported, 'prints live file output') // t.ok(output.indexOf('3 files') > -1, 'total size: files okay') // fs.unlinkSync(liveFile) // st.kill() // return true // }) // st.stderr.empty() // st.end() // }) // } // test.onFinish(function () { // rimraf.sync(path.join(fixtures, '.dat')) // }) ================================================ FILE: test/sync-remote.js ================================================ // var path = require('path') // var test = require('tape') // var rimraf = require('rimraf') // var spawn = require('./helpers/spawn.js') // var help = require('./helpers') // var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js')) // var baseTestDir = help.testFolder() // var shareDat // var syncDir // test('sync-remote - default opts', function (t) { // // cmd: dat sync // var key // help.shareFixtures({import: false}, function (_, fixturesDat) { // shareDat = fixturesDat // key = shareDat.key.toString('hex') // syncDir = path.join(baseTestDir, key) // makeClone(function () { // shareDat.importFiles(function () { // var cmd = dat + ' sync' // var st = spawn(t, cmd, {cwd: syncDir}) // st.stdout.match(function (output) { // var updated = output.indexOf('Files updated') > -1 // if (!updated) return false // var fileRe = new RegExp('3 files') // var bytesRe = new RegExp(/1\.\d{1,2} kB/) // key = help.matchLink(output) // t.ok(key, 'prints link') // t.ok(output.indexOf('dat-download-folder/' + key) > -1, 'prints dir') // t.ok(output.match(fileRe), 'total size: files okay') // t.ok(output.match(bytesRe), 'total size: bytes okay') // st.kill() // return true // }) // st.stderr.empty() // st.end() // }) // }) // }) // function makeClone (cb) { // var cmd = dat + ' clone ' + key // var st = spawn(t, cmd, {cwd: baseTestDir, end: false}) // st.stdout.match(function (output) { // var downloadFinished = output.indexOf('Download Finished') > -1 // if (!downloadFinished) return false // st.kill() // cb() // return true // }) // st.stderr.empty() // } // }) // test('sync-remote - shorthand sync', function (t) { // // cmd: dat sync // var cmd = dat + ' .' // var st = spawn(t, cmd, {cwd: syncDir}) // st.stdout.match(function (output) { // var syncing = output.indexOf('Syncing Dat Archive') > -1 // if (!syncing) return false // t.ok(help.matchLink(output), 'prints link') // st.kill() // return true // }) // st.stderr.empty() // st.end() // }) // test('sync-remote - dir arg', function (t) { // var cmd = dat + ' ' + syncDir // var st = spawn(t, cmd) // st.stdout.match(function (output) { // var syncing = output.indexOf('Syncing Dat Archive') > -1 // if (!syncing) return false // t.ok(help.matchLink(output), 'prints link') // st.kill() // return true // }) // st.stderr.empty() // st.end() // }) // test('close sharer', function (t) { // shareDat.close(function () { // rimraf.sync(path.join(shareDat.path, '.dat')) // t.end() // }) // }) // test.onFinish(function () { // rimraf.sync(baseTestDir) // }) ================================================ FILE: test/usage.js ================================================ var path = require('path') var test = require('tape') var spawn = require('./helpers/spawn.js') var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js')) var version = require('../package.json').version test('usage - prints usage', function (t) { var d = spawn(t, dat) d.stderr.match(function (output) { var usage = output.indexOf('dat ') > -1 if (!usage) return false return true }) d.end() }) test('usage - prints version', function (t) { var d = spawn(t, dat + ' -v') d.stderr.match(function (output) { var ver = output.indexOf(version) > -1 if (!ver) return false return true }) d.end() }) test('usage - also prints version', function (t) { var d = spawn(t, dat + ' -v') d.stderr.match(function (output) { var ver = output.indexOf(version) > -1 if (!ver) return false return true }) d.end() }) test('usage - help prints usage', function (t) { var d = spawn(t, dat + ' help') d.stderr.match(function (output) { var usage = output.indexOf('dat ') > -1 if (!usage) return false return true }) d.end() })