Repository: dat-ecosystem/dat
Branch: master
Commit: 14885d660adc
Files: 72
Total size: 159.0 KB
Directory structure:
gitextract_aa8sr91t/
├── .github/
│ ├── FUNDING.yml
│ └── issue_template.md
├── .gitignore
├── .travis.yml
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── LICENSE
├── README.md
├── appveyor.yml
├── bin/
│ └── cli.js
├── changelog.md
├── download.sh
├── index.js
├── package.json
├── package.sh
├── scripts/
│ └── auth-server.js
├── snap/
│ └── snapcraft.yaml
├── src/
│ ├── commands/
│ │ ├── auth/
│ │ │ ├── login.js
│ │ │ ├── logout.js
│ │ │ ├── register.js
│ │ │ └── whoami.js
│ │ ├── clone.js
│ │ ├── create.js
│ │ ├── doctor.js
│ │ ├── keys.js
│ │ ├── log.js
│ │ ├── publish.js
│ │ ├── pull.js
│ │ ├── status.js
│ │ ├── sync.js
│ │ └── unpublish.js
│ ├── extensions.js
│ ├── lib/
│ │ ├── archive.js
│ │ ├── discovery-exit.js
│ │ ├── download.js
│ │ ├── exit.js
│ │ ├── import-progress.js
│ │ ├── network.js
│ │ ├── selective-sync.js
│ │ ├── serve-http.js
│ │ └── stats.js
│ ├── parse-args.js
│ ├── registry.js
│ ├── ui/
│ │ ├── archive.js
│ │ ├── components/
│ │ │ ├── download.js
│ │ │ ├── import-progress.js
│ │ │ ├── network.js
│ │ │ ├── sources.js
│ │ │ └── warnings.js
│ │ ├── create.js
│ │ ├── elements/
│ │ │ ├── key.js
│ │ │ ├── pluralize.js
│ │ │ └── version.js
│ │ └── status.js
│ └── usage.js
└── test/
├── auth.js
├── clone.js
├── create.js
├── dat-node.js
├── doctor.js
├── fixtures/
│ ├── all_hour.csv
│ └── folder/
│ └── nested/
│ └── hello.txt
├── helpers/
│ ├── auth-server.js
│ ├── index.js
│ └── spawn.js
├── http.js
├── keys.js
├── pull.js
├── share.js
├── sync-owner.js
├── sync-remote.js
└── usage.js
================================================
FILE CONTENTS
================================================
================================================
FILE: .github/FUNDING.yml
================================================
open_collective: dat
================================================
FILE: .github/issue_template.md
================================================
<!--
Thanks for opening an issue! Please help us address your bug:
- The issue tracker is only for bugs and feature requests.
- Before reporting a bug, please make sure your version of Dat updated to the latest release.
- If you have a question or need general advice, ask us in our Chat: http://chat.datproject.org (#dat on IRC, freenode)
-->
I am reporting:
<!--- please select one and place above
- a bug or unexpected behavior
- general feedback
- feature request
DO NOT REPORT SECURITY ISSUES HERE
see security issue note below
-->
<!--
**Security Issue:**
Are you reporting a security issue that would impact general users? Please email us at security@datproject.org to report.
-->
# Bug Report
Please give us details about your installation to assist you. Run `dat -v` to see the version of Dat you are using.
* Operating system:
* Node Version:
* Dat Version:
### Expected behavior
<!-- What do you think should happen? -->
### Actual behavior
<!-- What actually happens? -->
### Debug Logs
<!-- If it is easy to reproduce your bug, please run with the debug output so we can see what is going on. Type `DEBUG=dat* <your-command>` to print debug logs. -->
```
```
================================================
FILE: .gitignore
================================================
node_modules
.DS_Store
tmp
.idea
data
yarn.lock
test/fixtures/.dat
test/fixtures/dat.json
test/**.db
test/.datrc-test
dist
builds
================================================
FILE: .travis.yml
================================================
language: node_js
node_js:
- 'lts/*'
- '12'
- 'node'
sudo: false
script:
- npm test
notifications:
irc:
channels:
- chat.freenode.net#datbots
template:
- '%{repository_slug} - %{commit_subject} - %{result} - %{build_url}'
skip_join: true
on_success: change
on_failure: always
before_deploy: npm run package
deploy:
provider: releases
api_key:
secure: GF+Ehh9kDu2m+KqSzciZRQmUfubnVGDEfxZKVX+psesKoxxDSq8/wkl7g1yR2H8DO0dg3lW8opbsKbfOOUWztyIfFxFukgwKIawUd7Krtr4XQLyywq49NdYARKP6bSxeEb8N3xVTo5fuq104KT0mMUB9di/iunsO/ITOzbCZyWE=
skip_cleanup: true
file_glob: true
file: dist/*
on:
repo: datproject/dat
node: '12'
tags: true
================================================
FILE: CODE_OF_CONDUCT.md
================================================
Our code of conduct is under review in [this repo](https://github.com/datproject/Code-of-Conduct) - please check it out and give us feedback!
================================================
FILE: CONTRIBUTING.md
================================================
# Welcome to Dat!
Please take a second to read over this before opening an issue. Providing complete information upfront will help us address any issue (and ship new features!) faster. Our time is limited, please respect it and create complete issues.
We are available to chat in IRC at #dat. You can join [via Gitter](https://gitter.im/datproject/discussions). You can also [view and search](https://botbot.me/freenode/dat/) chat logs.
We have a [faq](https://docs.datproject.org/faq) section on our docs that may address non-bug questions.
## Opening an Issue
Please read this section before opening a new issue.
`dat` is composed of many modules and this repository is just one of them. If you know your issue is with another module, we prefer you open it in that repository. There may be an exiting issue in another repository. If you aren't sure, then this is the perfect place.
Any new issues should be *actionable*. If your issue cannot be solved (and thus closed) please reconsider opening it in our discussion repository or rewording it.
### Bug Reports
A perfect bug report would have the following:
1. Summary of the issue you are experiencing.
2. Details on what versions of node and dat you have (`node -v` and `dat -v`).
3. A simple repeatable test case for us to run. Please try to run through it 2-3 times to ensure it is completely repeatable.
We would like to avoid issues that require a follow up questions to identify the bug. These follow ups are difficult to do unless we have a repeatable test case.
We can't all be perfect =). Do as much as you can and we'll try to help you with the rest. If we are slow to respond, a more complete bug report will help.
### Feature Requests
Feature requests are more than welcome. Please search exiting issues (open and closed) to make sure it is not a duplicate. A good feature request would have examples of how to use it, some detailed use cases, and any concerns or possible edge cases.
Keep in mind we have specific use cases we are building for (namely scientific data sharing). If, your feature request does not fit within that use case it may not be prioritized or implemented.
### General Discussion Issues
We prefer to be able to close issues in this repository, which does not lend itself to discussion type questions. Open discussion type issue in the [datproject/discussions](https://github.com/datproject/discussions/issues) repository.
## For Developers
Please read these guidelines if you are interested in contributing to Dat.
### Submitting pull requests
Before taking the time to code something, feel free to open an issue first proposing your idea to other contributors, that way you can get feedback on the idea before taking time to write precious code.
For any new functionality we like to see:
* unit tests so that we can improve long term maintenance and catch regressions in the future
* updates to the [change log](http://keepachangelog.com/) and relevant documentation
### For Collaborators
Make sure to get a `:thumbsup:`, `+1` or `LGTM` from another collaborator before merging a PR. If you aren't sure if a release should happen, open an issue.
Release process:
- make sure the tests pass
- Update changelog
- `npm version <major|minor|patch>`
- `git push && git push --tags`
- `npm publish`
### Development workflow
We use and write a lot of node modules and it introduces a bit of a learning curve when working on multiple modules simultaneously. There are lots of different and valid solutions to working on lots of modules at once, this is just one way.
#### Developing inside a node_modules folder
First make sure you are comfortable with [how require works](https://github.com/maxogden/art-of-node#how-require-works) in node.
We recommend creating a folder somewhere manually called `node_modules`. For example in `~/code/node_modules`. Clone all of your git copies of modules that you want to work on into here, so for example:
- `~/code/node_modules/dat`
- `~/code/node_modules/hyperdrive`
When you run `npm install` inside of `~/code/node_modules/dat`, dat will get its own copy of `hyperdrive` (one if its dependencies) inside `~/code/node_modules/dat/node_modules`. However, if you encounter a bug in hyperdrive that you need to fix, but you want to test your fix in dat, you want dat to use your git copy of hyperdrive at `~/code/node_modules/hyperdrive` and not the npm copy of hyperdrive at `~/code/node_modules/dat/node_modules/hyperdrive`.
How do you get dat to use the git copy of hyperdrive? Just delete the npm copy!
```
rm -rf ~/code/node_modules/dat/node_modules/hyperdrive
```
Now when you run dat, and it tries to `require('hyperdrive')` it first looks in its own `node_modules` folder at `~/code/node_modules/dat/node_modules` but doesnt find hyperdrive. So it goes up to `~/code/node_modules` and finds `hyperdrive` there and uses that one, your git copy.
If you want to switch back to an npm copy, just run `npm install` inside `~/code/node_modules/dat/` and npm will download any missing modules into `~/code/node_modules/dat/node_modules` but wont touch anything in `~/code/node_modules`.
This might seem a bit complicated at first, but is simple once you get the hang of it. Here are some rules to help you get started:
- Never make any meaningful edits to code inside an "npm-managed" node_modules folder (such as `~/code/node_modules/dat/node_modules`), because when you run `npm install` inside those folders it could inadvertently delete all of your edits when installing an updated copy of a module. This has happened to me many times, so I just always use my git copy and delete the npm copy (as described above) to make edits to a module.
- You should never need to run any npm commands in terminal when at your "manually managed"" node_modules folder at `~/code/node_modules`. Never running npm commands at that folder also prevents npm from accidentally erasing your git copies of modules
- The location of your "manually managed" node_modules folder should be somewhere isolated from your normal require path. E.g. if you put it at `~/node_modules`, then when you run `npm install dat` at `~/Desktop` npm might decide to erase your git copy of dat at `~/node_modules/dat` and replace it with a copy from npm, which could make you lose work. Putting your manually managed `node_modules` folder in a sub-folder like `~/code` gets it "out of the way" and prevents accidents like that from happening.
================================================
FILE: LICENSE
================================================
Copyright (c) 2015 Max Ogden. All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
================================================
FILE: README.md
================================================
More info on active projects and modules at [dat-ecosystem.org](https://dat-ecosystem.org/) <img src="https://i.imgur.com/qZWlO1y.jpg" width="30" height="30" />
---
# Dat
> npm install -g dat
Use `dat` command line to share files with version control, back up data to servers, browse remote files on demand, and automate long-term data preservation.
`dat` is the first application based upon the [Hypercore Protocol](https://github.com/hypercore-protocol), and drove the architectural design through iterative development between 2014 and 2017. There exists a large community around it.
[<img src="https://datproject.github.io/design/downloads/dat-logo.png" align="right" width="140">][Dat Project]
Have questions? Join our chat via IRC or Gitter:
[![#dat IRC channel on freenode][irc-badge]][irc-channel]
[![datproject/discussions][gitter-badge]][gitter-chat]
### Thanks to our financial supporters!
![OpenCollective](https://opencollective.com/dat/tiers/maintainer.svg?avatarHeight=36&width=600")
### Table of Contents
- [Installation](#installation)
- [Getting Started](#getting-started)
- [Using Dat](#usage)
- [Troubleshooting](#troubleshooting)
- [Javascript API](#js-api)
- [For Developers](#for-developers)
## Installation
Dat can be used as a command line tool or a javascript library:
* Install the `$ dat` CLI to use in the command line.
* [require('dat')][dat-node] - dat-node, a library for downloading and sharing dat archives in javascript apps.
### Installing the `$ dat` command line tool
The recommended way to install dat is through a single file binary distribution version of `dat` with the one line install command below. The binary includes a copy of node and dat packaged inside a single file, so you just have to download one file in order to start sharing data, with no other dependencies needed on your system:
```
wget -qO- https://raw.githubusercontent.com/datproject/dat/master/download.sh | bash
```
## Next version
Try the next version of dat! This version (14.0.0) is not compatible with older
versions (13.x) and below, and works on node v12.
```
npm install -g dat@next
```
Maintainers wanted!
#### NPM Prerequisites
* **Node**: You'll need to [install Node JS][install-node] before installing Dat. Dat needs `node` version 4 or above and `npm` installed. You can run `node -v` to check your version.
* **npm**: `npm` is installed with node. You can run `npm -v` to make sure it is installed.
Once you have `npm` ready, install `dat` from npm with the `--global, -g` option, `npm install -g dat`.
## Getting started
#### What is Dat?
Share, backup, and publish your filesystem. You can turn any folder on your computer into a dat. Dat scans your folder, allowing you to:
* Track your files with automatic version history.
* Share files with others over a secure peer to peer network.
* Automate live backups to external HDs or remote servers.
* Publish and share files with built in HTTP server.
Dat allows you to focus on the fun work without worrying about moving files around. **Secure**, **distributed**, **fast**.
* Documentation: [docs.datproject.org](https://docs.datproject.org)
* [Dat white paper](https://github.com/datprotocol/whitepaper/blob/master/dat-paper.pdf)
##### Desktop Applications
Rather not use the command line? Check out these options:
* [Beaker Browser] - An experimental p2p browser with built-in support for the Hypercore Protocol.
* [Dat Desktop](https://github.com/datproject/dat-desktop) - A desktop app to manage multiple dats on your desktop machine.
### JS Library
Add Dat to your `package.json`, `npm install dat --save`. Dat exports the [dat-node] API via `require('dat')`. Use it in your javascript applications! Dat Desktop and Dat command line both use dat-node to share and download dats.
Full API documentation is available in the [dat-node] repository on Github.
We have Dat installed, let's use it!
Dat's unique design works wherever you store your data. You can create a new dat from any folder on your computer.
A dat is some files from your computer and a `.dat` folder. Each dat has a unique `dat://` link. With your dat link, other users can download your files and live sync any updates.
### Sharing Data
You can start sharing your files with a single command. Unlike `git`, you do not have to initialize a repository first, `dat share` or simply `dat` will do that for you:
```
dat <dir>
```
Use `dat` to create a dat and sync your files from your computer to other users. Dat scans your files inside `<dir>`, creating metadata in `<dir>/.dat`. Dat stores the public link, version history, and file information inside the dat folder.
`dat sync` and `dat share` are aliases for the same command.
![share-gif]
### Downloading Data
```
dat dat://<link> <download-dir>
```
Use `dat` to download files from a remote computer sharing files with Dat. This will download the files from `dat://<link>` to your `<download-dir>`. The download exits after it completes but you can continue to update the files later after the clone is done. Use `dat pull` to update new files or `dat sync` to live sync changes.
`dat clone` is an alias for the same command.
![clone-gif]
### Misc Commands
A few other highlights. Run `dat help` to see the full usage guide.
* `dat create` or `dat init` - Create an empty dat and `dat.json` file.
* `dat log ~/data/dat-folder/` or `dat log dat://<key>` - view the history and metadata information for a dat.
### Quick Demos
To get started using Dat, you can try downloading a dat and then sharing a dat of your own.
#### Download Demo
We made a demo folder just for this exercise. Inside the demo folder is a `dat.json` file and a gif. We shared these files via Dat and now you can download them with our dat key!
Similar to git, you can download somebody's dat by running `dat clone <link>`. You can also specify the directory:
```
❯ dat clone dat://778f8d955175c92e4ced5e4f5563f69bfec0c86cc6f670352c457943666fe639 ~/Downloads/dat-demo
dat v13.5.0
Created new dat in /Users/joe/Downloads/dat-demo/.dat
Cloning: 2 files (1.4 MB)
2 connections | Download 614 KB/s Upload 0 B/s
dat sync complete.
Version 4
```
This will download our demo files to the `~/Downloads/dat-demo` folder. These files are being shared by a server over Dat (to ensure high availability) but you may connect to any number of users also hosting the content.
You can also also view the files online: [datbase.org/778f8d955175c92e4ced5e4f5563f69bfec0c86cc6f670352c457943666fe639](https://datbase.org/778f8d955175c92e4ced5e4f5563f69bfec0c86cc6f670352c457943666fe639/). datbase.org can download files over Dat and display them on HTTP as long as someone is hosting it. The website temporarily caches data for any visited links (do not view your dat on datbase.org if you do not want us to cache your data).
#### Sharing Demo
Dat can share files from your computer to anywhere. If you have a friend going through this demo with you, try sharing to them! If not we'll see what we can do.
Find a folder on your computer to share. Inside the folder can be anything, Dat can handle all sorts of files (Dat works with really big folders too!).
First, you can create a new dat inside that folder. Using the `dat create` command also walks us through making a `dat.json` file:
```
❯ dat create
Welcome to dat program!
You can turn any folder on your computer into a Dat.
A dat is a folder with some magic.
```
This will create a new (empty) dat. Dat will print a link, share this link to give others access to view your files.
Once we have our dat, run `dat <dir>` to scan your files and sync them to the network. Share the link with your friend to instantly start downloading files.
#### Bonus HTTP Demo
Dat makes it really easy to share live files on a HTTP server. This is a cool demo because we can also see how version history works! Serve dat files on HTTP with the `--http` option. For example, `dat --http`, serves your files to a HTTP website with live reloading and version history! This even works for dats you're downloading (add the `--sparse` option to only download files you select via HTTP). The default HTTP port is 8080.
*Hint: Use `localhost:8080/?version=10` to view a specific version.*
Get started using Dat today with the `share` and `clone` commands or read below for more details.
## Usage
The first time you run a command, a `.dat` folder is created to store the dat metadata.
Once a dat is created, you can run all the commands inside that folder, similar to git.
Dat keeps secret keys in the `~/.dat/secret_keys` folder. These are required to write to any dats you create.
#### Creating a dat & dat.json
```
dat create [<dir>]
```
The create command prompts you to make a `dat.json` file and creates a new dat. Import the files with sync or share.
Optionally bypass Title and Description prompt:
```sh
dat create --title "MY BITS" --description "are ready to synchronize! 😎"
```
Optionally bypass `dat.json` creation:
```sh
dat create --yes
dat create -y
```
### Sharing
The quickest way to get started sharing files is to `share`:
```
❯ dat
dat://3e830227b4b2be197679ff1b573cc85e689f202c0884eb8bdb0e1fcecbd93119
Sharing dat: 24 files (383 MB)
0 connections | Download 0 B/s Upload 0 B/s
Importing 528 files to Archive (165 MB/s)
[=-----------------------------------------] 3%
ADD: data/expn_cd.csv (403 MB / 920 MB)
```
```
dat [<dir>] [--no-import] [--no-watch]
```
Start sharing your dat archive over the network. It will import new or updated files since you last ran `create` or `sync`. Dat watches files for changes and imports updated files.
* Use `--no-import` to not import any new or updated files.
* Use `--no-watch` to not watch directory for changes. `--import` must be true for `--watch` to work.
#### Ignoring Files
By default, Dat will ignore any files in a `.datignore` file, similar to git. Each file should be separated by a newline. Dat also ignores all hidden folders and files. Supports pattern wildcards (`/*.png`) and directory-wildcards (`/**/cache`).
#### Selecting Files
By default, Dat will download all files. If you want to only download a subset, you can create a `.datdownload` file which downloads only the files and folders specified. Each should be separated by a newline.
### Downloading
Start downloading by running the `clone` command. This creates a folder, downloads the content and metadata, and a `.dat` folder inside. Once you started the download, you can resume at any time.
```
dat <link> [<dir>] [--temp]
```
Clone a remote dat archive to a local folder.
This will create a folder with the key name if no folder is specified.
#### Downloading via `dat.json` key
You can use a `dat.json` file to clone also. This is useful when combining Dat and git, for example. To clone a dat you can specify the path to a folder containing a `dat.json`:
```
git git@github.com:joehand/dat-clone-sparse-test.git
dat ./dat-clone-sparse-test
```
This will download the dat specified in the `dat.json` file.
#### Updating Downloaded Archives
Once a dat is clone, you can run either `dat pull` or `dat sync` in the folder to update the archive.
```
dat pull [<dir>]
```
Download latest files and keep connection open to continue updating as remote source is updated.
### Shortcut commands
* `dat <link> <dir>` will run `dat clone` for new dats or resume the existing dat in `<dir>`
* `dat <dir>` is the same as running `dat sync <dir>`
### Key Management & Moving dats
`dat keys` provides a few commands to help you move or backup your dats.
Writing to a dat requires the secret key, stored in the `~/.dat` folder. You can export and import these keys between dats. First, clone your dat to the new location:
* (original) `dat share`
* (duplicate) `dat clone <link>`
Then transfer the secret key:
* (original) `dat keys export` - copy the secret key printed out.
* (duplicate) `dat keys import` - this will prompt you for the secret key, paste it in here.
## Troubleshooting
We've provided some troubleshooting tips based on issues users have seen.
Please [open an issue][new-issue] or ask us in our [chat room][gitter-chat] if you need help troubleshooting and it is not covered here.
If you have trouble sharing/downloading in a directory with a `.dat` folder, try deleting it and running the command again.
#### Check Your Dat Version
Knowing the version is really helpful if you run into any bugs, and will help us troubleshoot your issue.
Check your Dat version:
```
dat -v
```
You should see the Dat semantic version printed, e.g. `14.0.0`.
### Installation Issues
#### Node & npm
To use the Dat command line tool you will need to have [node and npm installed][install-node-npm].
Make sure those are installed correctly before installing Dat.
You can check the version of each:
```
node -v
npm -v
```
#### Global Install
The `-g` option installs Dat globally, allowing you to run it as a command.
Make sure you installed with that option.
* If you receive an `EACCES` error, read [this guide][fixing-npm-permissions] on fixing npm permissions.
* If you receive an `EACCES` error, you may also install Dat with sudo: `sudo npm install -g dat`.
* Have other installation issues? Let us know, you can [open an issue][new-issue] or ask us in our [chat room][gitter-chat].
### Debugging Output
If you are having trouble with a specific command, run with the debug environment variable set to `dat` (and optionally also `dat-node`).
This will help us debug any issues:
```
DEBUG=dat,dat-node dat dat://<link> dir
```
### Networking Issues
Networking capabilities vary widely with each computer, network, and configuration.
Whenever you run Dat there are several steps to share or download files with peers:
1. Discovering Peers
2. Connecting to Peers
3. Sending & Receiving Data
With successful use, Dat will show `Connected to 1 peer` after connection.
If you never see a peer connected, your network may be restricting discovery or connection.
## JS API
You can use Dat in your javascript application:
```js
var Dat = require('dat')
Dat('/data', function (err, dat) {
// use dat
})
```
**[Read more][dat-node] about the JS usage provided via `dat-node`.**
## For Developers
Please see [guidelines on contributing] before submitting an issue or PR.
This command line library uses [dat-node] to create and manage the archives and networking.
If you'd like to build your own Dat application that is compatible with this command line tool, we suggest using dat-node.
### Installing from source
Clone this repository and in a terminal inside of the folder you cloned run this command:
```
npm link
```
This should add a `dat` command line command to your PATH.
Now you can run the `dat` command to try it out.
The contribution guide also has more tips on our [development workflow].
* `npm run test` to run tests
* `npm run auth-server` to run a local auth server for testing
## License
BSD-3-Clause
[Dat Project]: https://datproject.org
[Code for Science & Society]: https://codeforscience.org
[Dat white paper]: https://github.com/datproject/docs/blob/master/papers/dat-paper.pdf
[Dat Desktop]: https://docs.datproject.org/install#desktop-application
[Beaker Browser]: https://beakerbrowser.com
[registry server]: https://github.com/datproject/datbase
[share-gif]: https://raw.githubusercontent.com/datproject/docs/master/docs/assets/cli-share.gif
[clone-gif]: https://raw.githubusercontent.com/datproject/docs/master/docs/assets/cli-clone.gif
[Knight Foundation grant]: https://blog.datproject.org/2016/02/01/announcing-publicbits-org/
[dat-node]: https://github.com/datproject/dat-node
[dat-ignore]: https://github.com/joehand/dat-ignore
[new-issue]: https://github.com/datproject/dat/issues/new
[dat#503]: https://github.com/datproject/dat/issues/503
[install-node]: https://nodejs.org/en/download/
[install-node-npm]: https://docs.npmjs.com/getting-started/installing-node
[fixing-npm-permissions]: https://docs.npmjs.com/getting-started/fixing-npm-permissions
[guidelines on contributing]: https://github.com/datproject/dat/blob/master/CONTRIBUTING.md
[development workflow]: https://github.com/datproject/dat/blob/master/CONTRIBUTING.md#development-workflow
[travis-badge]: https://travis-ci.org/datproject/dat.svg?branch=master
[travis-build]: https://travis-ci.org/datproject/dat
[appveyor-badge]: https://ci.appveyor.com/api/projects/status/github/datproject/dat?branch=master&svg=true
[appveyor-build]: https://ci.appveyor.com/project/joehand/dat/branch/master
[npm-badge]: https://img.shields.io/npm/v/dat.svg
[npm-package]: https://npmjs.org/package/dat
[irc-badge]: https://img.shields.io/badge/irc%20channel-%23dat%20on%20freenode-blue.svg
[irc-channel]: https://webchat.freenode.net/?channels=dat
[gitter-badge]: https://badges.gitter.im/Join%20Chat.svg
[gitter-chat]: https://gitter.im/datproject/discussions
================================================
FILE: appveyor.yml
================================================
# Test against this version of Node.js
environment:
matrix:
- nodejs_version: "6"
- nodejs_version: "8"
- nodejs_version: "10"
# Install scripts. (runs after repo cloning)
install:
- ps: Install-Product node $env:nodejs_version
- npm install
test_script:
# Output useful info for debugging.
- node --version
- npm --version
- npm test
build: off
================================================
FILE: bin/cli.js
================================================
#!/usr/bin/env node
var subcommand = require('subcommand')
var debug = require('debug')('dat')
var usage = require('../src/usage')
var pkg = require('../package.json')
process.title = 'dat'
// Check node version to make sure we support
var NODE_VERSION_SUPPORTED = 4
var nodeMajorVer = process.version.match(/^v([0-9]+)\./)[1]
var invalidNode = nodeMajorVer < NODE_VERSION_SUPPORTED
if (invalidNode) exitInvalidNode()
else {
var notifier = require('update-notifier')
notifier({ pkg: pkg })
.notify({
defer: true,
isGlobal: true,
boxenOpts: {
align: 'left',
borderColor: 'green',
borderStyle: 'classic',
padding: 1,
margin: { top: 1, bottom: 1 }
}
})
}
if (debug.enabled) {
debug('Dat DEBUG mode engaged, enabling quiet mode')
}
var config = {
defaults: [
{ name: 'dir', abbr: 'd', help: 'set the directory for Dat' },
{ name: 'logspeed', default: 400 },
{ name: 'port', help: 'port to use for connections (default port: 3282 or first available)' },
{ name: 'utp', default: true, boolean: true, help: 'use utp for discovery' },
{ name: 'http', help: 'serve dat over http (default port: 8080)' },
{ name: 'debug', default: !!process.env.DEBUG && !debug.enabled, boolean: true },
{ name: 'quiet', default: debug.enabled, boolean: true }, // use quiet for dat debugging
{ name: 'sparse', default: false, boolean: true, help: 'download only requested data' },
{ name: 'up', help: 'throttle upload bandwidth (1024, 1kb, 2mb, etc.)' },
{ name: 'down', help: 'throttle download bandwidth (1024, 1kb, 2mb, etc.)' }
],
root: {
options: [
{
name: 'version',
boolean: true,
default: false,
abbr: 'v'
}
],
command: usage
},
none: syncShorthand,
commands: [
require('../src/commands/clone'),
require('../src/commands/create'),
require('../src/commands/log'),
require('../src/commands/keys'),
require('../src/commands/publish'),
require('../src/commands/pull'),
require('../src/commands/status'),
require('../src/commands/sync'),
require('../src/commands/unpublish'),
require('../src/commands/auth/register'),
require('../src/commands/auth/whoami'),
require('../src/commands/auth/logout'),
require('../src/commands/auth/login')
],
usage: {
command: showUsageOrRunExtension,
option: {
name: 'help',
abbr: 'h'
}
},
aliases: {
'init': 'create',
'share': 'sync'
},
// whitelist extensions for now
extensions: [
'store'
]
}
if (debug.enabled) {
debug('dat', pkg.version)
debug('node', process.version)
}
// Match Args + Run command
var match = subcommand(config)
match(alias(process.argv.slice(2)))
function alias (argv) {
var cmd = argv[0]
if (!config.aliases[cmd]) return argv
argv[0] = config.aliases[cmd]
return argv
}
// CLI Shortcuts
// Commands:
// dat <dat://key> [<dir>] - clone/sync a key
// dat <dir> - create dat + share a directory
// dat <extension>
function syncShorthand (opts) {
if (!opts._.length) return usage(opts)
debug('Sync shortcut command')
debug('Trying extension', opts._[0])
// First try extension
if (config.extensions.indexOf(opts._[0]) > -1) return require('../src/extensions')(opts)
var parsed = require('../src/parse-args')(opts)
// Download Key
if (parsed.key) {
// dat <dat://key> [<dir>] - clone/resume <link> in [dir]
debug('Clone sync')
opts.dir = parsed.dir || parsed.key // put in `process.cwd()/key` if no dir
opts.exit = opts.exit || false
return require('../src/commands/clone').command(opts)
}
// Sync dir
// dat <dir> - sync existing dat in {dir}
if (parsed.dir) {
opts.shortcut = true
debug('Share sync')
// Set default opts. TODO: use default opts in share
opts.watch = opts.watch || true
opts.import = opts.import || true
return require('../src/commands/sync').command(opts)
}
// All else fails, show usage
return usage(opts)
}
// This was needed so that we can show help messages from extensions
function showUsageOrRunExtension (opts, help, usageMessage) {
if (config.extensions.indexOf(opts._[0]) > -1) return require('../src/extensions')(opts)
usage(opts, help, usageMessage)
}
function exitInvalidNode () {
console.error('Node Version:', process.version)
console.error('Unfortunately, we only support Node >= v4. Please upgrade to use Dat.')
console.error('You can find the latest version at https://nodejs.org/')
process.exit(0)
}
================================================
FILE: changelog.md
================================================
# Change Log
All notable changes to this project will be documented in this file.
This project adheres to [Semantic Versioning](https://semver.org/).
## [Unreleased]
## 13.13.0
* `dat pull --exit=NN` exits after `NN` number of seconds, when there are no updates to sync.
## 13.9.0 - 2017-10-11
### Changed
* Use [datbase.org](https://datbase.org) as default registry (instead of datproject.org)
## 13.8.2 - 2017-09-28
### Fixed
* Error not being handled (https://github.com/datproject/dat/issues/838)
* Set `opts.debug` properly when using `DEBUG` that isn't `dat`.
* Move discovery key to option in `dat keys` (#869)
## 13.8.1 - 2017-08-04
### Fixes
* Error not being handled (https://github.com/datproject/dat/issues/838)
## 13.8.0 - 2017-08-04
With this release, we are adding an exciting feature that really showcases how powerful Dat is, selective sync. Using the CLI you can now specify which files you want to download either with an option or using the `.datdownload` file. `dat sync` will download and keep updated on the selected files. This means you can put large datasets into Dat but have control over what files you download where.
[Full release notes](https://github.com/datproject/dat/releases/tag/v13.8.0)
## Added
* Selective Sync (https://github.com/datproject/dat/pull/834)
* Key management (https://github.com/datproject/dat/pull/828)
## Changed
* Commands run faster via lazy required modules (https://github.com/datproject/dat/pull/821)
## 13.7.0 - 2017-06-28
## Added
* Throttling - sometimes Dat goes too fast, so you can limit the upload + download speeds. (https://github.com/datproject/dat/pull/806)
* Publish metadata to registry when publishing (https://github.com/datproject/dat/pull/812)
## Changed
* Use dat-node http support directly (https://github.com/datproject/dat/pull/817)
## Fixed
* Use npm package for registry testing.
## 13.6.0 - 2017-06-05
Full support for Dat registries! See our [full release notes](https://github.com/datproject/dat/releases/tag/v13.6.0).
### Added
* Improved support for public Dat registries (https://github.com/datproject/dat/pull/794)
* Add unpublish command
## 13.5.1 - 2017-05-30
### Changed
* Big documentation update!
* Force bump dat-node for a deletion bug that was a bit overzealous in deleting files (https://github.com/mafintosh/hyperdrive/issues/167).
## 13.5.0 - 2017-05-25
### Added
* Dat version number is printed in header (https://github.com/datproject/dat/pull/788)
* Add prompt and introduction to `dat create` command (https://github.com/datproject/dat/pull/782) and create dat.json file (https://github.com/datproject/dat/pull/765).
* Tell user if new `.dat` was initialized.
* Add `dat log` command to print archive history and size information (https://github.com/datproject/dat/pull/781).
* Use `require('dat')` to get `dat-node` JS API (https://github.com/datproject/dat/pull/778).
### Changed
* Default to upload true for `dat clone` and `dat pull`, enables better hole-punching (https://github.com/datproject/dat/pull/787).
### Fixed
* Make argument parsing more consistent across commands (https://github.com/datproject/dat/pull/789)
* Fix usage and help text (various).
## 13.4.1 - 2017-05-16
### Added
* Document sparse option in cli help
* add node/dat version to debug
### Changed
* Use share for shortcut (create new dat if not created)
### Fixed
* use exit option on clone shortcut if specified
* [various ui fixes](https://github.com/datproject/dat/pull/764)
## 13.4.0 - 2017-05-11
### Added
* Serve dat over http with `--http` option
## 13.3.0 - 2017-05-10
### Added
* Add `--sources` option for debugging network issues
## 13.2.0 - 2017-05-10
### Added
* Dat-* extensions ([#740](https://github.com/datproject/dat/pull/740))
* Ignore directories in import (dat-node v3.3.0)
## 13.1.1 - 2017-05-10
### Fixed
* Set directory for publish command
### Changed
* Improve `--show-key` help output
* Always show download progress bar and make language more clear.
## 13.1.0 - 2017-05-09
### Fixed
* Cleanup dat <link> shortcut + directory creation
* Check for any swarm.connecting before doing discovery failure.
### Added
* Check node version, fail for anything older than node v4 (#669)
* Add show-key option to display key on downloading cmds
* `dat status` command to show key, file count, dir size, and archive version
## 13.0.0 - 2017-05-08
### Changed
* Upgrade to Hyperdrive v8/9 (SLEEP archive format) and Dat-node v2/3. See [dat-node release docs](https://github.com/datproject/dat-node/releases/tag/v2.0.0) for more info.
* UI updates
## 12.0.3 - 2017-03-29
### Fixed
* Content progress for archives with history
* Change `process.title` to `dat` from `dat-next`
### Changed
* Use two decimals for content progress
## 12.0.2 - 2017-02-08
### Fixed
* Remove `hyperdrive-import-files` from dependencies (it is a dependency of `dat-node`). It was accidentally added.
* Always verify on read to avoid replication errors.
## 12.0.1 - 2017-02-07
### Fixed
* Files getting truncated and edited with bad characters - issue [#626](https://github.com/datproject/dat/issues/626) and [#623](https://github.com/datproject/dat/issues/623)
* Source files getting overwritten (issue [#628](https://github.com/datproject/dat/issues/628))
* Duplicate files getting imported
## 12.0.0 - 2017-02-06
Big new release! See the [release notes](https://github.com/datproject/dat/releases/tag/v12.0.0) on Github.
## 11.6.0 - 2016-11-16
### Removed
* webrtc support
### Fixed
* Fail gracefully if another dat is running in directory
* Handle `dat.open` errors
* Progress bar incorrectly showing 100% complete and 0 bytes
### Added
* Use graceful-fs to avoid EMFILE errors
## 11.5.5 - 2016-11-07
### Fixed
* Better download statistics using blocks instead of bytes
* Fix share stats on resuming without file changes
* Fix calculating size UI for large files
### Changed
* Update status logger. Uses [ansi-diff-stream](https://github.com/mafintosh/ansi-diff-stream) for updating CLI output now.
## 11.5.4 - 2016-10-28
### Changed
* Turn off `--watchFiles` by default
* Simplify progress UI
## 11.5.3 - 2016-10-28
### Fixed
* Fix `dat` command with no arguments
## 11.5.2 - 2016-10-24
### Fixed
* Fix `dat --doctor`
## 11.5.1 - 2016-10-24
### Fixed
* Resuming a folder previously shared fixed.
## 11.5.0 - 2016-10-20
### Added
* Accept dat.land links
* Allow `dat <dir>` to resume a downloaded link
### Fixed
* Improved error output for incorrect params
## 11.4.0 - 2016-10-06
### Added
* `--ignore-hidden` option. Ignores hidden files by default.
* `--signalhub` option to override default signalhub URL.
### Fixed
* Remove headless option from electron-webrtc. It is detected for us.
* `utp` is true by default
## 11.3.1 - 2016-09-21
### Fixed
* Use `--quiet` mode with `--debug` so output is easier to read.
## 11.3.0 - 2016-09-18
### Added
* `--webrtc` option. Uses electron-webrtc to run via webrtc.
## 11.2.0 - 2016-09-14
### Added
* `--temp` option. Uses memdb as database instead of `.dat` folder.
* Print message when download finishes telling user they can exit.
* Add option for turning off UTP
* Use dat-js module (includes using hyperdrive-import-files for appending)
### Fixed
* Download finished message not displayed when dat live updates
* Download speed removed when download is finished
## 11.1.2 - 2016-07-18
### Fixed
* Zero bytes total when downloading Dat with single file
## 11.1.1 - 2016-07-15
### Fixed
* Create download directory if doesn't exist
* Accept dat:// links for dat-desktop
* Throw error when two different dats are downloaded to same folder
## 11.1.0 - 2016-07-15
### Fixed
* Use yolowatch module for recursive live updates
* Improved stats for edge cases
* Print link with --quiet argument
* Better stat & progress output with hyperdrive/hypercore events
### Changed
* Simplified and clean up CLI output
* Improve modularity of library
* Move logger module into own npm package, status-logger
* Store key in .dat db without encoding as hex string (#498)
* upgrade to hyperdrive 7
### Removed
* List download option (will be re-added pending a hyperdrive update)
### Added
* Accept dat-encoding for 50 character links
## 11.0.2 - 2016-06-23
### Fixed
* Live mode with recursive adding files!
### Changed
* Separate yoloWatch to module
## 11.0.1 - 2016-06-20
### Fixed
* Create download directory if it doesn't exist
### Added
* Updated Docs
## 11.0.0 - 2016-06-17
### Added
* Live dat by default
* Added the `.dat` folder to store metadata and keys
* Resume dat share and download in existing .dat directory
* Store metadata using leveldb
* --list option on download to list files
* --exit option on download to close process on completion
### Changed
* New proposed RC2 API
* --static option change to --snapshot
* Use Hyperdrive-archive-swarm module for swarm
### Removed
* --seed option, stays open by default now
* --no-color option
* --append option, append by default now
## 10.1.1 - 2016-06-09
### Fixed
* Fix file count on live share
* Fix total percentage on share
## 10.1.0 - 2016-06-08
### Changed
* Show progress in CLI output
## 10.0.2 - 2016-06-07
### Fixed
* Fix --static sharing
* Fix --doctor
## 10.0.1 - 2016-06-06
### Fixed
* Share argument
* Argument bugs
## 10.0.0 - 2016-06-06
### Added
* Live sharing!
### Changed
* Update to hyperdrive 6.0
* Update API to RC2 candidate
## 9.x.x and earlier
These refer to the pre-1.0 versions of dat and are omitted.
================================================
FILE: download.sh
================================================
#!/bin/bash
# gets latest dat release zip for platform and extracts runnable binary into ~/.dat
# usage: wget -qO- https://raw.githubusercontent.com/datproject/dat/master/bin/install.sh | bash
# based on https://github.com/jpillora/installer/blob/master/scripts/download.sh
DAT_DIR="$HOME/.dat/releases"
function cleanup {
rm -rf $DAT_DIR/tmp.zip > /dev/null
}
function fail {
cleanup
msg=$1
echo "============"
echo "Error: $msg" 1>&2
exit 1
}
function install {
# bash check
[ ! "$BASH_VERSION" ] && fail "Please use bash instead"
GET=""
if which curl > /dev/null; then
GET="curl"
GET="$GET --fail -# -L"
elif which wget > /dev/null; then
GET="wget"
GET="$GET -qO-"
else
fail "neither wget/curl are installed"
fi
case `uname -s` in
Darwin) OS="macos";;
Linux) OS="linux";;
*) fail "unsupported os: $(uname -s)";;
esac
if uname -m | grep 64 > /dev/null; then
ARCH="x64"
else
fail "only arch x64 is currently supported for single file install. please use npm instead. your arch is: $(uname -m)"
fi
echo "Fetching latest Dat release version from GitHub"
LATEST=$($GET -qs https://api.github.com/repos/datproject/dat/releases/latest | grep tag_name | head -n 1 | cut -d '"' -f 4);
mkdir -p $DAT_DIR || fail "Could not create directory $DAT_DIR, try manually downloading zip and extracting instead."
cd $DAT_DIR
RELEASE="dat-${LATEST:1}-${OS}-${ARCH}"
URL="https://github.com/datproject/dat/releases/download/${LATEST}/${RELEASE}.zip"
which unzip > /dev/null || fail "unzip is not installed"
echo "Downloading $URL"
bash -c "$GET $URL" > $DAT_DIR/tmp.zip || fail "download failed"
unzip -o -qq $DAT_DIR/tmp.zip || fail "unzip failed"
BIN="$DAT_DIR/$RELEASE/dat"
chmod +x $BIN || fail "chmod +x failed"
cleanup
printf "Dat $LATEST has been downloaded successfully. Execute it with this command:\n\n${BIN}\n\nAdd it to your PATH with this command (add this to .bash_profile/.bashrc):\n\nexport PATH=\"\$PATH:$DAT_DIR/$RELEASE\"\n"
}
install
================================================
FILE: index.js
================================================
module.exports = require('dat-node')
================================================
FILE: package.json
================================================
{
"name": "dat",
"version": "14.0.3",
"description": "Dat is the package manager for data. Easily share and version control data.",
"keywords": [
"dat",
"dat protocol",
"hyperdrive",
"decentralized",
"file sharing"
],
"main": "index.js",
"bin": {
"dat": "bin/cli.js"
},
"scripts": {
"auth-server": "DEBUG=* node scripts/auth-server.js",
"install-precommit": "echo ./node_modules/.bin/standard > .git/hooks/pre-commit && chmod +x .git/hooks/pre-commit",
"standard": "standard",
"deps": "dependency-check . && dependency-check . --extra --no-dev",
"test": "standard && npm run deps && tape 'test/*.js'",
"test-only": "tape 'test/*.js'",
"package": "rm -rf builds && npm run pkg && ./package.sh",
"pkg": "pkg package.json -o builds/dat"
},
"repository": {
"type": "git",
"url": "git://github.com/datproject/dat.git"
},
"author": "Dat Project",
"license": "BSD-3-Clause",
"bugs": {
"url": "https://github.com/datproject/dat/issues"
},
"homepage": "https://datproject.org",
"directories": {
"test": "tests"
},
"dependencies": {
"bytes": "^3.1.0",
"chalk": "^2.4.2",
"cli-truncate": "^1.0.0",
"dat-encoding": "^5.0.1",
"dat-json": "^1.0.3",
"dat-link-resolve": "^2.3.0",
"dat-log": "^2.0.0",
"dat-node": "^4.0.0",
"dat-registry": "^4.0.1",
"debug": "^4.0.0",
"neat-log": "^3.1.0",
"prettier-bytes": "^1.0.3",
"progress-string": "^1.2.1",
"prompt": "^1.0.0",
"pump": "^3.0.0",
"rimraf": "^2.7.1",
"speedometer": "^1.1.0",
"subcommand": "^2.1.1",
"throttle": "^1.0.3",
"update-notifier": "^2.3.0"
},
"devDependencies": {
"cross-zip-cli": "^1.0.0",
"dependency-check": "^3.4.1",
"hypercore": "^6.25.2",
"mkdirp": "^0.5.4",
"node-fetch": "^2.6.1",
"pkg": "^4.4.4",
"random-access-memory": "^3.1.1",
"recursive-readdir-sync": "^1.0.6",
"standard": "^12.0.0",
"tape": "^4.13.2",
"tape-spawn": "^1.4.2",
"temporary-directory": "^1.0.2"
},
"pkg": {
"assets": [
"./node_modules/utp-native/prebuilds/**",
"./node_modules/blake2b-wasm/blake2b.wasm",
"./node_modules/siphash24/siphash24.wasm"
],
"targets": [
"node10-linux-x64",
"node10-macos-x64",
"node10-win-x64"
]
}
}
================================================
FILE: package.sh
================================================
#!/usr/bin/env sh
# couldnt figure out undocumented 'output template' mode for pkg so wrote this
# also need to include .node files until pkg supports including them in binary
NODE_ABI="node.napi"
VERSION=$(node -pe "require('./package.json').version")
rm -rf dist
mkdir dist
mkdir builds/dat-$VERSION-linux-x64
mkdir builds/dat-$VERSION-macos-x64
mkdir builds/dat-$VERSION-win-x64
mv builds/dat-linux builds/dat-$VERSION-linux-x64/dat
mv builds/dat-macos builds/dat-$VERSION-macos-x64/dat
mv builds/dat-win.exe builds/dat-$VERSION-win-x64/dat.exe
cp node_modules/utp-native/prebuilds/linux-x64/$NODE_ABI.node builds/dat-$VERSION-linux-x64/
cp node_modules/utp-native/prebuilds/darwin-x64/$NODE_ABI.node builds/dat-$VERSION-macos-x64/
cp node_modules/utp-native/prebuilds/win32-x64/$NODE_ABI.node builds/dat-$VERSION-win-x64/
cp LICENSE builds/dat-$VERSION-linux-x64/
cp LICENSE builds/dat-$VERSION-macos-x64/
cp LICENSE builds/dat-$VERSION-win-x64/
cp README.md builds/dat-$VERSION-linux-x64/README
cp README.md builds/dat-$VERSION-macos-x64/README
cp README.md builds/dat-$VERSION-win-x64/README
cd builds
../node_modules/.bin/cross-zip dat-$VERSION-linux-x64 ../dist/dat-$VERSION-linux-x64.zip
../node_modules/.bin/cross-zip dat-$VERSION-macos-x64 ../dist/dat-$VERSION-macos-x64.zip
../node_modules/.bin/cross-zip dat-$VERSION-win-x64 ../dist/dat-$VERSION-win-x64.zip
rm -rf builds
# now travis will upload the 3 zips in dist to the release
================================================
FILE: scripts/auth-server.js
================================================
var createServer = require('../tests/helpers/auth-server')
createServer(process.env.PORT || 8888, function (err, server, closeServer) {
if (err) throw err
process.on('exit', close)
process.on('SIGINT', close)
function close (cb) {
closeServer(function () {
process.exit()
})
}
})
================================================
FILE: snap/snapcraft.yaml
================================================
name: dat
version: '13.11.4'
summary: Share & live sync files anywhere via command line
description: |
Use Dat command line to share files with version control,
back up data to servers, browse remote files on demand,
and automate long-term data preservation.
grade: 'stable'
confinement: 'strict'
apps:
dat:
command: dat
plugs:
- home
- network
- network-bind
- removable-media
parts:
dat:
source: https://github.com/datproject/dat.git
source-tag: 'v13.11.4'
plugin: nodejs
node-engine: 10.9.0
================================================
FILE: src/commands/auth/login.js
================================================
module.exports = {
name: 'login',
command: login,
help: [
'Login to a Dat registry server',
'Usage: dat login [<registry>]',
'',
'Publish your dats so other users can discovery them.',
'Please register before trying to login.'
].join('\n'),
options: [
{
name: 'server',
help: 'Your Dat registry server (must be registered to login).'
}
]
}
function login (opts) {
var prompt = require('prompt')
var output = require('neat-log/output')
var chalk = require('chalk')
var Registry = require('../../registry')
if (opts._[0]) opts.server = opts._[0]
var welcome = output(`
Welcome to ${chalk.green(`dat`)} program!
Login to publish your dats.
`)
console.log(welcome)
var schema = {
properties: {
server: {
description: chalk.magenta('Dat registry'),
default: opts.server || 'datbase.org',
required: true
},
email: {
description: chalk.magenta('Email'),
message: 'Email required',
required: true
},
password: {
description: chalk.magenta('Password'),
message: 'Password required',
required: true,
hidden: true,
replace: '*'
}
}
}
prompt.override = opts
prompt.message = ''
prompt.start()
prompt.get(schema, function (err, results) {
if (err) return exitErr(err)
opts.server = results.server
makeRequest(results)
})
function makeRequest (user) {
var client = Registry(opts)
client.login({
email: user.email,
password: user.password
}, function (err, resp, body) {
if (err && err.message) return exitErr(err.message)
else if (err) return exitErr(err.toString())
console.log(output(`
Logged you in to ${chalk.green(opts.server)}!
Now you can publish dats and share:
* Run ${chalk.green(`dat publish`)} to publish a dat!
* View & Share your dats at ${opts.server}
`))
process.exit(0)
})
}
}
function exitErr (err) {
console.error(err)
process.exit(1)
}
================================================
FILE: src/commands/auth/logout.js
================================================
module.exports = {
name: 'logout',
command: logout,
help: [
'Logout from current Dat registry server',
'Usage: dat logout [<registry>]',
'',
'Specify server if you want to from non-active other server.',
'Check active server with `dat whoami`.'
].join('\n'),
options: [
{
name: 'server',
help: 'Server to log out of. Defaults to active login.'
}
]
}
function logout (opts) {
var chalk = require('chalk')
var Registry = require('../../registry')
if (opts._[0]) opts.server = opts._[0]
var client = Registry(opts)
var whoami = client.whoami()
if (!whoami || !whoami.token) return exitErr('Not currently logged in to that server.')
client.logout(function (err) {
if (err) return exitErr(err)
console.log(`Logged out of ${chalk.green(whoami.server)}`)
process.exit(0)
})
}
function exitErr (err) {
console.error(err)
process.exit(1)
}
================================================
FILE: src/commands/auth/register.js
================================================
module.exports = {
name: 'register',
command: register,
help: [
'Register with a public Dat registry',
'Usage: dat register [<registry>]',
'',
'Register with datbase.org or other registries to publish your dats.'
].join('\n'),
options: [
{
name: 'server',
help: 'Your Dat registry.'
}
]
}
function register (opts) {
var prompt = require('prompt')
var output = require('neat-log/output')
var chalk = require('chalk')
var Registry = require('../../registry')
// TODO: check if logged in?
if (opts._[0]) opts.server = opts._[0]
var welcome = output(`
Welcome to ${chalk.green(`dat`)} program!
Create a new account with a Dat registry.
`)
console.log(welcome)
var schema = {
properties: {
server: {
description: chalk.magenta('Dat registry'),
default: opts.server || 'datbase.org',
required: true
},
username: {
description: chalk.magenta('Username'),
message: 'Username required',
required: true
},
email: {
description: chalk.magenta('Email'),
message: 'Email required',
required: true
},
password: {
description: chalk.magenta('Password'),
message: 'Password required',
required: true,
hidden: true,
replace: '*'
}
}
}
prompt.override = opts
prompt.message = ''
prompt.start()
prompt.get(schema, function (err, results) {
if (err) return exitErr(err)
opts.server = results.server
makeRequest(results)
})
function makeRequest (user) {
var client = Registry(opts)
client.register({
email: user.email,
username: user.username,
password: user.password
}, function (err) {
if (err && err.message) return exitErr(err.message)
else if (err) return exitErr(err.toString())
console.log(output(`
Created account on ${chalk.green(opts.server)}!
Login to start publishing: ${chalk.green(`dat login`)}
`))
process.exit(0)
})
}
}
function exitErr (err) {
console.error(err)
process.exit(1)
}
================================================
FILE: src/commands/auth/whoami.js
================================================
module.exports = {
name: 'whoami',
command: whoami,
help: [
'Get login information',
'Usage: dat login [<registry>]',
'',
'Get information for active registry or specify your registry.'
].join('\n'),
options: [
{
name: 'server',
help: 'Server to get login information for. Defaults to active login.'
}
]
}
function whoami (opts) {
var output = require('neat-log/output')
var chalk = require('chalk')
var Registry = require('../../registry')
if (opts._[0]) opts.server = opts._[0]
var client = Registry(opts)
var login = client.whoami()
if (!login || !login.token) {
if (!opts.server) return exitErr('No login information found.')
return exitErr('No login information found for that server.')
}
console.log(output(`
Your active Dat registry information:
---
${chalk.green(login.server)}
Email: ${login.email}
Username: ${login.username}
---
Change your registry by logging in again:
${chalk.dim.green('dat login <registry-url>')}
`))
process.exit(0)
}
function exitErr (err) {
console.error(err)
process.exit(1)
}
================================================
FILE: src/commands/clone.js
================================================
module.exports = {
name: 'clone',
command: clone,
help: [
'Clone a remote Dat archive',
'',
'Usage: dat clone <link> [download-folder]'
].join('\n'),
options: [
{
name: 'empty',
boolean: false,
default: false,
help: 'Do not download files by default. Files must be synced manually.'
},
{
name: 'upload',
boolean: true,
default: true,
help: 'announce your address on link (improves connection capability) and upload data to other downloaders.'
},
{
name: 'show-key',
boolean: true,
default: false,
abbr: 'k',
help: 'print out the dat key'
}
]
}
function clone (opts) {
var fs = require('fs')
var path = require('path')
var rimraf = require('rimraf')
var Dat = require('dat-node')
var linkResolve = require('dat-link-resolve')
var neatLog = require('neat-log')
var archiveUI = require('../ui/archive')
var trackArchive = require('../lib/archive')
var discoveryExit = require('../lib/discovery-exit')
var onExit = require('../lib/exit')
var parseArgs = require('../parse-args')
var debug = require('debug')('dat')
var parsed = parseArgs(opts)
opts.key = parsed.key || opts._[0] // pass other links to resolver
opts.dir = parsed.dir
opts.showKey = opts['show-key'] // using abbr in option makes printed help confusing
opts.sparse = opts.empty
debug('clone()')
// cmd: dat /path/to/dat.json (opts.key is path to dat.json)
if (fs.existsSync(opts.key)) {
try {
opts.key = getDatJsonKey()
} catch (e) {
debug('error reading dat.json key', e)
}
}
debug(Object.assign({}, opts, { key: '<private>', _: null })) // don't show key
var neat = neatLog(archiveUI, { logspeed: opts.logspeed, quiet: opts.quiet, debug: opts.debug })
neat.use(trackArchive)
neat.use(discoveryExit)
neat.use(onExit)
neat.use(function (state, bus) {
if (!opts.key) return bus.emit('exit:warn', 'key required to clone')
state.opts = opts
var createdDirectory = null // so we can delete directory if we get error
// Force these options for clone command
opts.exit = (opts.exit !== false)
// opts.errorIfExists = true // TODO: do we want to force this?
linkResolve(opts.key, function (err, key) {
if (err && err.message.indexOf('Invalid key') === -1) return bus.emit('exit:error', 'Could not resolve link')
else if (err) return bus.emit('exit:warn', 'Link is not a valid Dat link.')
opts.key = key
createDir(opts.key, function () {
bus.emit('key', key)
runDat()
})
})
function createDir (key, cb) {
debug('Checking directory for clone')
// Create the directory if it doesn't exist
// If no dir is specified, we put dat in a dir with name = key
if (!opts.dir) opts.dir = key
if (!Buffer.isBuffer(opts.dir) && typeof opts.dir !== 'string') {
return bus.emit('exit:error', 'Directory path must be a string or Buffer')
}
fs.access(opts.dir, fs.F_OK, function (err) {
if (!err) {
createdDirectory = false
return cb()
}
debug('No existing directory, creating it.')
createdDirectory = true
fs.mkdir(opts.dir, cb)
})
}
function runDat () {
Dat(opts.dir, opts, function (err, dat) {
if (err && err.name === 'ExistsError') return bus.emit('exit:warn', 'Existing archive in this directory. Use pull or sync to update.')
if (err) {
if (createdDirectory) rimraf.sync(dat.path)
return bus.emit('exit:error', err)
}
if (dat.writable) return bus.emit('exit:warn', 'Archive is writable. Cannot clone your own archive =).')
state.dat = dat
state.title = 'Cloning'
bus.emit('dat')
bus.emit('render')
})
}
})
function getDatJsonKey () {
var datPath = opts.key
var stat = fs.lstatSync(datPath)
if (stat.isDirectory()) datPath = path.join(datPath, 'dat.json')
if (!fs.existsSync(datPath) || path.basename(datPath) !== 'dat.json') {
if (stat.isFile()) throw new Error('must specify existing dat.json file to read key')
throw new Error('directory must contain a dat.json')
}
debug('reading key from dat.json:', datPath)
return JSON.parse(fs.readFileSync(datPath, 'utf8')).url
}
}
================================================
FILE: src/commands/create.js
================================================
module.exports = {
name: 'create',
command: create,
help: [
'Create an empty dat and dat.json',
'',
'Usage: dat create [directory]'
].join('\n'),
options: [
{
name: 'yes',
boolean: true,
default: false,
abbr: 'y',
help: 'Skip dat.json creation.'
},
{
name: 'title',
help: 'the title property for dat.json'
},
{
name: 'description',
help: 'the description property for dat.json'
}
]
}
function create (opts) {
var path = require('path')
var fs = require('fs')
var Dat = require('dat-node')
var output = require('neat-log/output')
var DatJson = require('dat-json')
var prompt = require('prompt')
var chalk = require('chalk')
var parseArgs = require('../parse-args')
var debug = require('debug')('dat')
debug('dat create')
if (!opts.dir) {
opts.dir = parseArgs(opts).dir || process.cwd()
}
var welcome = `Welcome to ${chalk.green(`dat`)} program!`
var intro = output(`
You can turn any folder on your computer into a Dat.
A Dat is a folder with some magic.
Your dat is ready!
We will walk you through creating a 'dat.json' file.
(You can skip dat.json and get started now.)
Learn more about dat.json: ${chalk.blue(`https://github.com/datprotocol/dat.json`)}
${chalk.dim('Ctrl+C to exit at any time')}
`)
var outro
// Force certain options
opts.errorIfExists = true
console.log(welcome)
Dat(opts.dir, opts, function (err, dat) {
if (err && err.name === 'ExistsError') return exitErr('\nArchive already exists.\nYou can use `dat sync` to update.')
if (err) return exitErr(err)
outro = output(`
Created empty Dat in ${dat.path}/.dat
Now you can add files and share:
* Run ${chalk.green(`dat share`)} to create metadata and sync.
* Copy the unique dat link and securely share it.
${chalk.blue(`dat://${dat.key.toString('hex')}`)}
`)
if (opts.yes) return done()
console.log(intro)
var datjson = DatJson(dat.archive, { file: path.join(opts.dir, 'dat.json') })
fs.readFile(path.join(opts.dir, 'dat.json'), 'utf-8', function (err, data) {
if (err || !data) return doPrompt()
data = JSON.parse(data)
debug('read existing dat.json data', data)
doPrompt(data)
})
function doPrompt (data) {
if (!data) data = {}
var schema = {
properties: {
title: {
description: chalk.magenta('Title'),
default: data.title || '',
// pattern: /^[a-zA-Z\s\-]+$/,
// message: 'Name must be only letters, spaces, or dashes',
required: false
},
description: {
description: chalk.magenta('Description'),
default: data.description || ''
}
}
}
prompt.override = { title: opts.title, description: opts.description }
prompt.message = '' // chalk.green('> ')
// prompt.delimiter = ''
prompt.start()
prompt.get(schema, writeDatJson)
function writeDatJson (err, results) {
if (err) return exitErr(err) // prompt error
if (!results.title && !results.description) return done()
datjson.create(results, done)
}
}
function done (err) {
if (err) return exitErr(err)
console.log(outro)
}
})
function exitErr (err) {
if (err && err.message === 'canceled') {
console.log('')
console.log(outro)
process.exit(0)
}
console.error(err)
process.exit(1)
}
}
================================================
FILE: src/commands/doctor.js
================================================
module.exports = {
name: 'doctor',
help: [
'Call the Doctor! Runs two tests:',
' 1. Check if you can connect to a peer on a public server.',
' 2. Gives you a link to test direct peer connections.',
'',
'Usage: dat doctor [<link>]'
].join('\n'),
options: [],
command: function (opts) {
var doctor = require('dat-doctor')
opts.peerId = opts._[0]
doctor(opts)
}
}
================================================
FILE: src/commands/keys.js
================================================
module.exports = {
name: 'keys',
command: keys,
help: [
'View & manage dat keys',
'',
'Usage:',
'',
' dat keys view dat key and discovery key',
' dat keys export export dat secret key',
' dat keys import import dat secret key to make a dat writable',
''
].join('\n'),
options: [
{
name: 'discovery',
boolean: true,
default: false,
help: 'Print Discovery Key'
}
]
}
function keys (opts) {
var Dat = require('dat-node')
var parseArgs = require('../parse-args')
var debug = require('debug')('dat')
debug('dat keys')
if (!opts.dir) {
opts.dir = parseArgs(opts).dir || process.cwd()
}
opts.createIfMissing = false // keys must always be a resumed archive
Dat(opts.dir, opts, function (err, dat) {
if (err && err.name === 'MissingError') return exit('Sorry, could not find a dat in this directory.')
if (err) return exit(err)
run(dat, opts)
})
}
function run (dat, opts) {
var subcommand = require('subcommand')
var prompt = require('prompt')
var config = {
root: {
command: function () {
console.log(`dat://${dat.key.toString('hex')}`)
if (opts.discovery) console.log(`Discovery key: ${dat.archive.discoveryKey.toString('hex')}`)
process.exit()
}
},
commands: [
{
name: 'export',
command: function foo (args) {
if (!dat.writable) return exit('Dat must be writable to export.')
console.log(dat.archive.metadata.secretKey.toString('hex'))
}
},
{
name: 'import',
command: function bar (args) {
if (dat.writable) return exit('Dat is already writable.')
importKey()
}
}
]
}
subcommand(config)(process.argv.slice(3))
function importKey () {
// get secret key & write
var schema = {
properties: {
key: {
pattern: /^[a-z0-9]{128}$/,
message: 'Use `dat keys export` to get the secret key (128 character hash).',
hidden: true,
required: true,
description: 'dat secret key'
}
}
}
prompt.message = ''
prompt.start()
prompt.get(schema, function (err, data) {
if (err) return done(err)
var secretKey = data.key
if (typeof secretKey === 'string') secretKey = Buffer.from(secretKey, 'hex')
// Automatically writes the metadata.ogd file
dat.archive.metadata._storage.secretKey.write(0, secretKey, done)
})
function done (err) {
if (err) return exit(err)
console.log('Successful import. Dat is now writable.')
exit()
}
}
}
function exit (err) {
if (err) {
console.error(err)
process.exit(1)
}
process.exit(0)
}
================================================
FILE: src/commands/log.js
================================================
module.exports = {
name: 'log',
help: [
'View history and information about a dat',
'',
'Usage: dat log [dir|link]'
].join('\n'),
options: [
{
name: 'live',
boolean: true,
default: false,
help: 'View live updates to history.'
}
],
command: function (opts) {
var log = require('dat-log')
log(opts)
}
}
================================================
FILE: src/commands/publish.js
================================================
module.exports = {
name: 'publish',
command: publish,
help: [
'Publish your dat to a Dat registry',
'Usage: dat publish [<registry>]',
'',
'By default it will publish to your active registry.',
'Specify the server to change where the dat is published.'
].join('\n'),
options: [
{
name: 'server',
help: 'Publish dat to this registry. Defaults to active login.'
}
]
}
function publish (opts) {
var path = require('path')
var Dat = require('dat-node')
var encoding = require('dat-encoding')
var output = require('neat-log/output')
var prompt = require('prompt')
var chalk = require('chalk')
var DatJson = require('dat-json')
var xtend = Object.assign
var Registry = require('../registry')
if (!opts.dir) opts.dir = process.cwd()
if (opts._[0]) opts.server = opts._[0]
if (!opts.server) opts.server = 'datbase.org' // nicer error message if not logged in
var client = Registry(opts)
var whoami = client.whoami()
if (!whoami || !whoami.token) {
var loginErr = output(`
Welcome to ${chalk.green(`dat`)} program!
Publish your dats to ${chalk.green(opts.server)}.
${chalk.bold('Please login before publishing')}
${chalk.green('dat login')}
New to ${chalk.green(opts.server)} and need an account?
${chalk.green('dat register')}
Explore public dats at ${chalk.blue('datbase.org/explore')}
`)
return exitErr(loginErr)
}
opts.createIfMissing = false // publish must always be a resumed archive
Dat(opts.dir, opts, function (err, dat) {
if (err && err.name === 'MissingError') return exitErr('No existing dat in this directory. Create a dat before publishing.')
else if (err) return exitErr(err)
dat.joinNetwork() // join network to upload metadata
var datjson = DatJson(dat.archive, { file: path.join(dat.path, 'dat.json') })
datjson.read(publish)
function publish (_, data) {
// ignore datjson.read() err, we'll prompt for name
// xtend dat.json with opts
var datInfo = xtend({
name: opts.name,
url: 'dat://' + encoding.toStr(dat.key), // force correct url in publish? what about non-dat urls?
title: opts.title,
description: opts.description
}, data)
var welcome = output(`
Publishing dat to ${chalk.green(opts.server)}!
`)
console.log(welcome)
if (datInfo.name) return makeRequest(datInfo)
prompt.message = ''
prompt.start()
prompt.get({
properties: {
name: {
description: chalk.magenta('dat name'),
pattern: /^[a-zA-Z0-9-]+$/,
message: `A dat name can only have letters, numbers, or dashes.\n Like ${chalk.bold('cool-cats-12meow')}`,
required: true
}
}
}, function (err, results) {
if (err) return exitErr(err)
datInfo.name = results.name
makeRequest(datInfo)
})
}
function makeRequest (datInfo) {
console.log(`Please wait, '${chalk.bold(datInfo.name)}' will soon be ready for its great unveiling...`)
client.dats.create(datInfo, function (err, resp, body) {
if (err) {
if (err.message) {
if (err.message === 'timed out') {
return exitErr(output(`${chalk.red('\nERROR: ' + opts.server + ' could not connect to your computer.')}
Troubleshoot here: ${chalk.green('https://docs.datproject.org/troubleshooting#networking-issues')}
`))
}
var str = err.message.trim()
if (str === 'jwt expired') return exitErr(`Session expired, please ${chalk.green('dat login')} again`)
return exitErr('ERROR: ' + err.message) // node error
}
// server response errors
return exitErr('ERROR: ' + err.toString())
}
if (body.statusCode === 400) return exitErr(new Error(body.message))
datjson.write(datInfo, function (err) {
if (err) return exitErr(err)
// TODO: write published url to dat.json (need spec)
var msg = output(`
We ${body.updated === 1 ? 'updated' : 'published'} your dat!
${chalk.blue.underline(`${opts.server}/${whoami.username}/${datInfo.name}`)}
`)// TODO: get url back? it'd be better to confirm link than guess username/datname structure
console.log(msg)
if (body.updated === 1) {
console.log(output(`
${chalk.dim.green('Cool fact #21')}
${opts.server} will live update when you are sharing your dat!
You only need to publish again if your dat link changes.
`))
} else {
console.log(output(`
Remember to use ${chalk.green('dat share')} before sharing.
This will make sure your dat is available.
`))
}
process.exit(0)
})
})
}
})
}
function exitErr (err) {
console.error(err)
process.exit(1)
}
================================================
FILE: src/commands/pull.js
================================================
module.exports = {
name: 'pull',
command: pull,
help: [
'Pull updates from a cloned Dat archive',
'',
'Usage: dat pull'
].join('\n'),
options: [
{
name: 'exit',
boolean: false,
help: 'exit after specified number of seconds, to give the dat network time to find updates. (default: 12)'
},
{
name: 'upload',
boolean: true,
default: true,
help: 'announce your address on link (improves connection capability) and upload data to other downloaders.'
},
{
name: 'selectFromFile',
boolean: false,
default: '.datdownload',
help: 'Sync only the list of selected files or directories in the given file.',
abbr: 'select-from-file'
},
{
name: 'select',
boolean: false,
default: false,
help: 'Sync only the list of selected files or directories.'
},
{
name: 'show-key',
boolean: true,
default: false,
abbr: 'k',
help: 'print out the dat key'
}
]
}
function pull (opts) {
var Dat = require('dat-node')
var neatLog = require('neat-log')
var archiveUI = require('../ui/archive')
var trackArchive = require('../lib/archive')
var selectiveSync = require('../lib/selective-sync')
var discoveryExit = require('../lib/discovery-exit')
var onExit = require('../lib/exit')
var parseArgs = require('../parse-args')
var debug = require('debug')('dat')
debug('dat pull')
if (!opts.dir) {
var parsed = parseArgs(opts)
opts.key = parsed.key
opts.dir = parsed.dir || process.cwd()
}
opts.showKey = opts['show-key'] // using abbr in option makes printed help confusing
// Force these options for pull command
opts.createIfMissing = false
// If --exit is specified without a number of seconds, default to 12
if (opts.exit) {
opts.exit = typeof opts.exit === 'number'
? opts.exit
: 12
}
var neat = neatLog(archiveUI, { logspeed: opts.logspeed, quiet: opts.quiet, debug: opts.debug })
neat.use(trackArchive)
neat.use(discoveryExit)
neat.use(onExit)
neat.use(function (state, bus) {
state.opts = opts
selectiveSync(state, opts)
Dat(opts.dir, opts, function (err, dat) {
if (err && err.name === 'MissingError') return bus.emit('exit:warn', 'No existing archive in this directory. Use clone to download a new archive.')
if (err) return bus.emit('exit:error', err)
if (dat.writable) return bus.emit('exit:warn', 'Archive is writable. Cannot pull your own archive.')
state.dat = dat
bus.emit('dat')
bus.emit('render')
})
})
}
================================================
FILE: src/commands/status.js
================================================
module.exports = {
name: 'status',
command: status,
help: [
'Get information on about the Dat in a directory.',
'',
'Usage: dat status'
].join('\n'),
options: []
}
function status (opts) {
var Dat = require('dat-node')
var neatLog = require('neat-log')
var statusUI = require('../ui/status')
var onExit = require('../lib/exit')
var parseArgs = require('../parse-args')
var debug = require('debug')('dat')
debug('dat status')
if (!opts.dir) {
opts.dir = parseArgs(opts).dir || process.cwd()
}
opts.createIfMissing = false // sync must always be a resumed archive
var neat = neatLog(statusUI, { logspeed: opts.logspeed, quiet: opts.quiet, debug: opts.debug })
neat.use(onExit)
neat.use(function (state, bus) {
state.opts = opts
Dat(opts.dir, opts, function (err, dat) {
if (err && err.name === 'MissingError') return bus.emit('exit:warn', 'Sorry, could not find a dat in this directory.')
if (err) return bus.emit('exit:error', err)
state.dat = dat
var stats = dat.trackStats()
if (stats.get().version === dat.version) return exit()
stats.on('update', function () {
if (stats.get().version === dat.version) return exit()
})
function exit () {
bus.render()
process.exit(0)
}
})
})
}
================================================
FILE: src/commands/sync.js
================================================
module.exports = {
name: 'sync',
command: sync,
help: [
'Sync a Dat archive with the network',
'Watch and import file changes',
'',
'Usage: dat sync'
].join('\n'),
options: [
{
name: 'import',
boolean: true,
default: true,
help: 'Import files from the directory to the database (when writable).'
},
{
name: 'ignoreHidden',
boolean: true,
default: true,
abbr: 'ignore-hidden'
},
{
name: 'selectFromFile',
boolean: false,
default: '.datdownload',
help: 'Sync only the list of selected files or directories in the given file.',
abbr: 'select-from-file'
},
{
name: 'select',
boolean: false,
default: false,
help: 'Sync only the list of selected files or directories.'
},
{
name: 'watch',
boolean: true,
default: true,
help: 'Watch for changes and import updated files (Dat Writable).'
},
{
name: 'show-key',
boolean: true,
default: true,
abbr: 'k',
help: 'Print out the dat key.'
}
]
}
function sync (opts) {
var Dat = require('dat-node')
var neatLog = require('neat-log')
var archiveUI = require('../ui/archive')
var selectiveSync = require('../lib/selective-sync')
var trackArchive = require('../lib/archive')
var onExit = require('../lib/exit')
var parseArgs = require('../parse-args')
var debug = require('debug')('dat')
debug('dat sync')
var parsed = parseArgs(opts)
opts.key = parsed.key
opts.dir = parsed.dir || process.cwd()
opts.showKey = opts['show-key'] // using abbr in option makes printed help confusing
// TODO: if dat-store running, add this dat to the local store and then exit = true
opts.exit = false
var neat = neatLog(archiveUI, { logspeed: opts.logspeed, quiet: opts.quiet, debug: opts.debug })
neat.use(trackArchive)
neat.use(onExit)
neat.use(function (state, bus) {
state.opts = opts
selectiveSync(state, opts)
Dat(opts.dir, opts, function (err, dat) {
if (err && err.name === 'IncompatibleError') return bus.emit('exit:warn', 'Directory contains incompatible dat metadata. Please remove the .dat folder in this directory.')
if (err) return bus.emit('exit:error', err)
state.dat = dat
bus.emit('dat')
bus.emit('render')
})
})
}
================================================
FILE: src/commands/unpublish.js
================================================
module.exports = {
name: 'unpublish',
command: unpublish,
options: [
{
name: 'server',
help: 'Unpublish dat from this Registry.'
},
{
name: 'confirm',
default: false,
boolean: true,
abbr: 'y',
help: 'Confirm you want to unpublish'
}
]
}
function unpublish (opts) {
var prompt = require('prompt')
var path = require('path')
var Dat = require('dat-node')
var output = require('neat-log/output')
var chalk = require('chalk')
var DatJson = require('dat-json')
var Registry = require('../registry')
if (opts._[0]) opts.server = opts._[0]
if (!opts.dir) opts.dir = process.cwd() // run in dir for `dat unpublish`
var client = Registry(opts)
var whoami = client.whoami()
if (!whoami || !whoami.token) {
var loginErr = output(`
Welcome to ${chalk.green(`dat`)} program!
${chalk.bold('You must login before unpublishing.')}
${chalk.green('dat login')}
`)
return exitErr(loginErr)
}
opts.createIfMissing = false // unpublish dont try to create new one
Dat(opts.dir, opts, function (err, dat) {
if (err) return exitErr(err)
// TODO better error msg for non-existing archive
if (!dat.writable) return exitErr('Sorry, you can only publish a dat that you created.')
var datjson = DatJson(dat.archive, { file: path.join(dat.path, 'dat.json') })
datjson.read(function (err, data) {
if (err) return exitErr(err)
if (!data.name) return exitErr('Try `dat unpublish <name>` with this dat, we are having trouble reading it.')
confirm(data.name)
})
})
function confirm (name) {
console.log(`Unpublishing '${chalk.bold(name)}' from ${chalk.green(whoami.server)}.`)
prompt.message = ''
prompt.colors = false
prompt.start()
prompt.get([{
name: 'sure',
description: 'Are you sure? This cannot be undone. [y/n]',
pattern: /^[a-zA-Z\s-]+$/,
message: '',
required: true
}], function (err, results) {
if (err) return console.log(err.message)
if (results.sure === 'yes' || results.sure === 'y') makeRequest(name)
else exitErr('Cancelled.')
})
}
function makeRequest (name) {
client.dats.delete({ name: name }, function (err, resp, body) {
if (err && err.message) exitErr(err.message)
else if (err) exitErr(err.toString())
if (body.statusCode === 400) return exitErr(new Error(body.message))
console.log(`Removed your dat from ${whoami.server}`)
process.exit(0)
})
}
}
function exitErr (err) {
console.error(err)
process.exit(1)
}
================================================
FILE: src/extensions.js
================================================
var debug = require('debug')('dat')
var os = require('os')
module.exports = runExtension
function runExtension (opts) {
debug('Trying Extenion', opts._[0])
var extName = opts._.shift()
trySpawn(function () {
console.error('We could not run the extension. Please make sure it is installed:')
console.error(`npm install -g dat-${extName}`)
process.exit(1)
})
function trySpawn (cb) {
var spawn = require('child_process').spawn
var name = 'dat-' + extName
if (os.platform() === 'win32') {
name += '.cmd'
}
var child = spawn(name, process.argv.splice(3))
child.stdout.pipe(process.stdout)
child.stderr.pipe(process.stderr)
child.on('error', function (err) {
if (err.code === 'ENOENT') return cb()
throw err
})
child.on('close', function (code) {
process.exit(code)
})
}
}
================================================
FILE: src/lib/archive.js
================================================
var debug = require('debug')('dat')
var path = require('path')
var EventEmitter = require('events').EventEmitter
var doImport = require('./import-progress')
var stats = require('./stats')
var network = require('./network')
var download = require('./download')
var serve = require('./serve-http')
module.exports = function (state, bus) {
state.warnings = state.warnings || []
bus.once('dat', function () {
state.writable = state.dat.writable
state.joinNetwork = !(state.joinNetwork === false)
stats(state, bus)
if (state.joinNetwork) network(state, bus)
if (state.opts.http) serve(state, bus)
if (state.writable && state.opts.import) doImport(state, bus)
else if (state.opts.sparse) selectiveSync(state, bus)
else download(state, bus)
if (state.dat.archive.content) return bus.emit('archive:content')
state.dat.archive.once('content', function () {
bus.emit('archive:content')
})
})
bus.once('archive:content', function () {
state.hasContent = true
})
}
function selectiveSync (state, bus) {
var archive = state.dat.archive
debug('sparse mode. downloading metadata')
var emitter = new EventEmitter()
function download (entry) {
debug('selected', entry)
archive.stat(entry, function (err, stat) {
if (err) return state.warnings.push(err.message)
if (stat.isDirectory()) downloadDir(entry, stat)
if (stat.isFile()) downloadFile(entry, stat)
})
}
function downloadDir (dirname, stat) {
debug('downloading dir', dirname)
archive.readdir(dirname, function (err, entries) {
if (err) return bus.emit('exit:error', err)
entries.forEach(function (entry) {
emitter.emit('download', path.join(dirname, entry))
})
})
}
function downloadFile (entry, stat) {
var start = stat.offset
var end = stat.offset + stat.blocks
state.selectedByteLength += stat.size
bus.emit('render')
if (start === 0 && end === 0) return
debug('downloading', entry, start, end)
archive.content.download({ start, end }, function () {
debug('success', entry)
})
}
emitter.on('download', download)
if (state.opts.selectedFiles) state.opts.selectedFiles.forEach(download)
if (state.opts.empty) {
archive.metadata.update(function () {
return bus.emit('exit:warn', `Dat successfully created in empty mode. Download files using pull or sync.`)
})
}
archive.on('update', function () {
debug('archive update')
bus.emit('render')
})
}
================================================
FILE: src/lib/discovery-exit.js
================================================
var output = require('neat-log/output')
module.exports = discoveryExit
function discoveryExit (state, bus) {
bus.once('network:callback', checkExit)
function checkExit () {
if (state.dat.network.connections || !state.opts.exit) return
if (state.dat.network.connecting) return setTimeout(checkExit, 500) // wait to see if any connections resolve
var msg = output(`
Dat could not find any connections for that link.
There may not be any sources online.
Ensure that everyone is using the latest version, using dat -v
`)
bus.emit('exit:warn', msg)
}
}
================================================
FILE: src/lib/download.js
================================================
var debug = require('debug')('dat')
var xtend = Object.assign
module.exports = trackDownload
function trackDownload (state, bus) {
if (state.hasContent) return track()
bus.once('archive:content', track)
function track () {
var archive = state.dat.archive
state.download = xtend({
modified: false,
nsync: false
}, {})
archive.content.on('clear', function () {
debug('archive clear')
state.download.modified = true
})
archive.content.on('download', function (index, data) {
state.download.modified = true
})
archive.on('syncing', function () {
debug('archive syncing')
state.download.nsync = false
})
archive.on('sync', function () {
debug('archive sync', state.stats.get())
state.download.nsync = true
// if we are supposed to exit, do so if we've pulled changes or have given the network the desired wait time
if (state.opts.exit) {
if (state.download.modified) {
return exit()
} else {
var delayInMilliseconds = 1000 * state.opts.exit
setTimeout(exit, delayInMilliseconds)
}
}
if (state.dat.archive.version === 0) {
// TODO: deal with this.
// Sync sometimes fires early when it should wait for update.
}
bus.emit('render')
})
archive.on('update', function () {
debug('archive update')
bus.emit('render')
})
function exit () {
if (state.stats.get().version !== archive.version) {
return state.stats.on('update', exit)
}
state.exiting = true
bus.render()
process.exit(0)
}
}
}
================================================
FILE: src/lib/exit.js
================================================
module.exports = onExit
function onExit (state, bus) {
bus.on('exit:error', onError)
bus.on('exit:warn', function (err) {
onError(err, true)
})
bus.on('exit', function () {
state.exiting = true
bus.render()
process.exit()
})
function onError (err, clear) {
if (clear) bus.clear()
console.error(err)
process.exit(1)
}
}
================================================
FILE: src/lib/import-progress.js
================================================
var xtend = Object.assign
module.exports = trackImport
function trackImport (state, bus) {
if (state.dat) return track()
bus.once('dat', track)
function track () {
var progress = state.dat.importFiles(state.opts, function (err) {
if (err) return bus.emit('exit:error', err)
state.importer.fileImport = null
state.exiting = true
bus.emit('render')
})
state.importer = xtend({
importedBytes: 0,
count: progress.count,
liveImports: [],
indexSpeed: progress.indexSpeed
}, progress)
bus.emit('dat:importer')
var counting = setInterval(function () {
// Update file count in progress counting (for big dirs)
bus.emit('render')
}, state.opts.logspeed)
progress.on('count', function (count) {
clearInterval(counting)
state.count = count
state.count.done = true
bus.emit('render')
})
progress.on('del', function (src, dst) {
if (src.live) state.importer.liveImports.push({ src: src, dst: dst, type: 'del' })
})
progress.on('put', function (src, dst) {
if (src.live) state.importer.liveImports.push({ src: src, dst: dst, type: 'put' })
if (src.stat.isDirectory()) return
state.importer.fileImport = {
src: src,
dst: dst,
progress: 0,
type: 'put'
}
bus.emit('render')
})
progress.on('put-data', function (chunk, src, dst) {
state.importer.fileImport.progress += chunk.length
if (!src.live) state.importer.importedBytes += chunk.length // don't include live in total
state.importer.indexSpeed = progress.indexSpeed
bus.emit('render')
})
}
}
================================================
FILE: src/lib/network.js
================================================
var bytes = require('bytes').parse
var speed = require('speedometer')
var throttle = require('throttle')
var pump = require('pump')
var debug = require('debug')('dat')
var xtend = Object.assign
module.exports = trackNetwork
function trackNetwork (state, bus) {
if (state.dat) return track()
bus.once('dat', track)
function track () {
var opts = state.opts
if (state.opts.up || state.opts.down) {
opts = xtend({}, opts, {
connect: function (local, remote) {
var streams = [local, remote, local]
if (state.opts.up) streams.splice(1, 0, throttle(bytes(state.opts.up)))
if (state.opts.down) streams.splice(-1, 0, throttle(bytes(state.opts.down)))
pump(streams)
}
})
}
var network = state.dat.joinNetwork(opts, function () {
bus.emit('network:callback')
})
network.on('error', function (err) {
if (err.code === 'EADDRINUSE') {
if (opts.port) {
bus.emit('exit:warn', `Specified port (${opts.port}) in use. Please use another port.`)
} else {
debug(err.message + ' trying random port')
}
} else {
debug('network error:', err.message)
// TODO return bus.emit('exit:error', err)
}
})
state.network = xtend(network, state.network)
bus.emit('dat:network')
network.on('connection', function (conn, info) {
bus.emit('render')
conn.on('close', function () {
bus.emit('render')
})
})
if (state.opts.sources) trackSources()
if (state.stats) return trackSpeed()
bus.once('dat:stats', trackSpeed)
function trackSpeed () {
setInterval(function () {
bus.emit('render')
}, state.opts.logspeed)
}
function trackSources () {
state.sources = state.sources || {}
network.on('connection', function (conn, info) {
var id = info.id.toString('hex')
var peerSpeed = speed()
state.sources[id] = info
state.sources[id].speed = peerSpeed()
state.sources[id].getProgress = function () {
// TODO: how to get right peer from archive.content?
// var remote = conn.feeds[1].remoteLength
// // state.dat.archive.content.sources[0].feed.id.toString('hex')
// if (!remote) return
// return remote / dat.archive.content.length
}
conn.feeds.map(function (feed) {
feed.stream.on('data', function (data) {
state.sources[id].speed = peerSpeed(data.length)
bus.emit('render')
})
feed.stream.on('error', function (err) {
state.sources[id].error = err
})
})
bus.emit('render')
conn.on('close', function () {
state.sources[id].speed = 0
state.sources[id].closed = true
bus.emit('render')
})
})
}
}
}
================================================
FILE: src/lib/selective-sync.js
================================================
var fs = require('fs')
var path = require('path')
module.exports = function (state, opts) {
// selective sync stuff
var parsing = opts.selectFromFile !== '.datdownload' ? opts.selectFromFile : path.join(opts.dir, '.datdownload')
opts.selectedFiles = parseFiles(parsing)
if (opts.select && typeof opts.select === 'string') opts.selectedFiles = opts.select.split(',')
if (opts.selectedFiles) {
state.title = 'Syncing'
state.selectedByteLength = 0
opts.sparse = true
}
return state
}
function parseFiles (input) {
var parsed = null
try {
if (fs.statSync(input).isFile()) {
parsed = fs.readFileSync(input).toString().trim().split(/\r?\n/)
}
} catch (err) {
if (err && !err.name === 'ENOENT') {
console.error(err)
process.exit(1)
}
}
return parsed
}
================================================
FILE: src/lib/serve-http.js
================================================
module.exports = runHttp
function runHttp (state, bus) {
if (state.dat) return serve()
bus.once('dat', serve)
function serve () {
var port = (typeof state.opts.http === 'boolean') ? 8080 : state.opts.http
var server = state.dat.serveHttp({ port: port })
server.on('listening', function () {
state.http = { port: port, listening: true }
bus.emit('render')
})
}
}
================================================
FILE: src/lib/stats.js
================================================
var xtend = Object.assign
module.exports = trackStats
function trackStats (state, bus) {
if (state.dat) return track()
bus.once('dat', track)
function track () {
var stats = state.dat.trackStats(state.opts)
state.stats = xtend(stats, state.stats)
stats.on('update', function () {
bus.emit('stats:update')
bus.emit('render')
})
bus.emit('stats')
}
}
================================================
FILE: src/parse-args.js
================================================
var fs = require('fs')
var path = require('path')
var encoding = require('dat-encoding')
module.exports = function (opts) {
// dat [<cmd>] arg1 arg2 [options]
// parse args without options from opts._
// return parsed { dir, key }
var parsed = {
key: opts.key || null,
dir: opts.dir || null // process.cwd() ?
}
// dat [<cmd>]
if (!opts._.length) return parsed
// dat [<cmd>] arg1 arg2
// arg1 = key
// arg2 = dir
if (opts._.length === 2) {
parsed.key = opts._[0]
parsed.dir = opts._[1]
return parsed
}
// dat [<cmd>] arg
// arg = dir or key
// First, check if key
try {
parsed.key = encoding.toStr(opts._[0])
return parsed
} catch (err) {
if (err && err.message !== 'Invalid key') {
// catch non-key errors
console.error(err)
process.exit(1)
}
}
try {
var stat = fs.statSync(opts._[0])
if (stat.isFile()) {
parsed.dir = path.resolve(path.dirname(opts._[0]))
} else {
parsed.dir = opts._[0]
}
} catch (err) {
if (err && !err.name === 'ENOENT') {
console.error(err)
process.exit(1)
}
}
return parsed
}
================================================
FILE: src/registry.js
================================================
var xtend = Object.assign
var RegistryClient = require('dat-registry')
module.exports = function (opts) {
var townshipOpts = {
server: opts.server,
config: {
filepath: opts.config // defaults to ~/.datrc via dat-registry
}
}
var defaults = {
// xtend doesn't overwrite when key is present but undefined
// If we want a default, make sure it's not going to passed as undefined
}
var options = xtend(defaults, townshipOpts)
return RegistryClient(options)
}
================================================
FILE: src/ui/archive.js
================================================
var path = require('path')
var output = require('neat-log/output')
var pretty = require('prettier-bytes')
var chalk = require('chalk')
var downloadUI = require('./components/download')
var importUI = require('./components/import-progress')
var warningsUI = require('./components/warnings')
var networkUI = require('./components/network')
var sourcesUI = require('./components/sources')
var keyEl = require('./elements/key')
var pluralize = require('./elements/pluralize')
var version = require('./elements/version')
var pkg = require('../../package.json')
module.exports = archiveUI
function archiveUI (state) {
if (!state.dat) return 'Starting Dat program...'
if (!state.writable && !state.hasContent) return 'Connecting to dat network...'
if (!state.warnings) state.warnings = []
var dat = state.dat
var stats = dat.stats.get()
var title = (state.dat.resumed) ? '' : `Created new dat in ${dat.path}${path.sep}.dat\n`
var progressView
if (state.writable || state.opts.showKey) {
title += `${keyEl(dat.key)}\n`
}
if (state.title) title += state.title
else if (state.writable) title += 'Sharing dat'
else title += 'Downloading dat'
if (state.opts.sparse) title += `: ${state.opts.selectedFiles.length} ${pluralize('file', state.opts.selectedFiles.length)} (${pretty(state.selectedByteLength)})`
else if (stats.version > 0) title += `: ${stats.files} ${pluralize('file', stats.file)} (${pretty(stats.byteLength)})`
else if (stats.version === 0) title += ': (empty archive)'
if (state.http && state.http.listening) title += `\nServing files over http at http://localhost:${state.http.port}`
if (!state.writable) {
progressView = downloadUI(state)
} else {
if (state.opts.import) {
progressView = importUI(state)
} else {
progressView = 'Not importing files.' // TODO: ?
}
}
return output(`
${version(pkg.version)}
${title}
${state.joinNetwork ? '\n' + networkUI(state) : ''}
${progressView}
${state.opts.sources ? sourcesUI(state) : ''}
${state.warnings ? warningsUI(state) : ''}
${state.exiting ? 'Exiting the Dat program...' : chalk.dim('Ctrl+C to Exit')}
`)
}
================================================
FILE: src/ui/components/download.js
================================================
var output = require('neat-log/output')
var bar = require('progress-string')
module.exports = networkUI
function networkUI (state) {
var stats = state.stats.get()
var download = state.download
if (!stats || !download) return ''
var title = 'Downloading updates...'
var downBar = makeBar()
if (download.nsync) {
if (state.opts.exit && state.dat.archive.version === 0) {
return 'dat synced. There is no content in this archive.'
}
if (state.opts.exit && download.modified) {
return `dat sync complete.\nVersion ${stats.version}`
}
if (!download.modified && state.opts.exit) {
title = `dat already in sync, waiting for updates.`
} else {
title = `dat synced, waiting for updates.`
}
}
if (typeof state.opts.exit === 'number') {
title = `dat synced, exiting in ${state.opts.exit} seconds.`
}
if (!stats.downloaded || !stats.length) {
return '' // no metadata yet
}
return output(`
${title}
${downBar(stats.downloaded)}
`)
function makeBar () {
var total = stats.length
return bar({
total: total,
style: function (a, b) {
return `[${a}${b}] ${(100 * stats.downloaded / total).toFixed(2)}%`
}
})
}
}
================================================
FILE: src/ui/components/import-progress.js
================================================
var output = require('neat-log/output')
var pretty = require('prettier-bytes')
var bar = require('progress-string')
var cliTruncate = require('cli-truncate')
module.exports = importUI
function importUI (state) {
var watch = state.opts.watch
var importState = state.importer
var indexSpeed = importState.indexSpeed ? `(${pretty(importState.indexSpeed)}/s)` : ''
if (importState.count && !importState.count.done) {
// dry run in progress
if (!importState.count.files) return 'Checking for file updates...'
return output(`
Metadata created for ${importState.putDone.files} of ${importState.count.files} files ${indexSpeed}
(Calculating file count...)
${fileImport(importState.fileImport)}
`)
} else if (importState.putDone.files >= importState.count.files) {
// Initial import done
if (!watch) return 'Archive metadata updated for all files.'
return liveImport()
}
var total = importState.count.bytes
var totalBar = bar({
total: total,
style: function (a, b) {
return `[${a}${b}] ${(100 * importState.importedBytes / total).toFixed(0)}%`
}
})
return output(`
Creating metadata for ${importState.count.files} files ${indexSpeed}
${totalBar(importState.importedBytes)}
${fileImport(importState.fileImport)}
`)
function liveImport () {
// Live import
var imports = importState.liveImports.slice(1).slice(-7)
return output(`
Watching for file updates
${imports.reverse().map(function (file) { return fileImport(file) }).join('\n')}
`)
}
function fileImport (file) {
if (!file) return ''
if (file.type === 'del') return `DEL: ${file.src.name}`
var total = file.src.stat.size
var name = file.dst.name.substr(1) // remove '/' at start
var size
// >500 mb show progress
if (total < 5e8 || !file.progress) size = `(${pretty(total)})`
else size = `(${pretty(file.progress)} / ${pretty(total)})`
return output(`
ADD: ${cliTruncate(name, process.stdout.columns - 7 - size.length, { position: 'start' })} ${size}
`)
}
}
================================================
FILE: src/ui/components/network.js
================================================
var output = require('neat-log/output')
var pretty = require('prettier-bytes')
var pluralize = require('../elements/pluralize')
module.exports = networkUI
function networkUI (state) {
var network = state.network
var stats = state.stats
if (!network) return ''
var peers = stats.peers.total || 0
// var complete = stats.peers.complete
return output(`
${peers} ${pluralize('connection', peers)} ${speedUI()}
`)
function speedUI () {
var output = '| '
var speed = state.stats.network
var upSpeed = speed.uploadSpeed || 0
var downSpeed = speed.downloadSpeed || 0
output += `Download ${pretty(downSpeed)}/s`
output += ` Upload ${pretty(upSpeed)}/s `
return output
}
}
================================================
FILE: src/ui/components/sources.js
================================================
var output = require('neat-log/output')
var pretty = require('prettier-bytes')
var makeBar = require('progress-string')
module.exports = peersUI
function peersUI (state) {
if (!state.network) return ''
if (Object.keys(state.sources).length === 0) return ''
var peers = state.sources
// var stats = state.stats
// var peerCount = stats.peers.total || 0
// var complete = stats.peers.complete
var info = Object.keys(peers).map(function (id, i) {
return peerUI(peers[id], i)
}).join('\n')
return `\n${info}\n`
function peerUI (peer, i) {
var progress = peer.getProgress()
var bar = makeBar({
total: 100,
style: function (a, b) {
return `[${a}${b}] ${(progress).toFixed(2)}%`
}
})
var theBar = progress ? bar(progress) : '' // progress bar todo
return output(`
[${i}] ${peer.closed ? 'CLOSED' : peer.type}: ${peer.host}:${peer.port} ${pretty(peer.speed)}/s
${peer.error ? peer.error : theBar}
`)
}
}
================================================
FILE: src/ui/components/warnings.js
================================================
var chalk = require('chalk')
module.exports = function (state) {
var warning = ''
state.warnings.forEach(function (message) {
warning += `${chalk.yellow(`Warning: ${message}`)}\n`
})
return warning
}
================================================
FILE: src/ui/create.js
================================================
var output = require('neat-log/output')
var pretty = require('prettier-bytes')
var chalk = require('chalk')
var importUI = require('./components/import-progress')
var keyEl = require('./elements/key')
var pluralize = require('./elements/pluralize')
module.exports = createUI
function createUI (state) {
if (!state.dat) {
return output(`
Creating a Dat! Add information to your dat.json file:
`)
}
var dat = state.dat
var stats = dat.stats.get()
var title = '\n'
var progressView
var exitMsg = `
Your dat is created! Run ${chalk.green('dat sync')} to share:
${keyEl(dat.key)}
`
if (!state.opts.import) {
// set exiting right away
state.exiting = true
}
if (!state.exiting) {
// Only show key if not about to exit
title = `${keyEl(dat.key)}\n`
}
if (state.title) title += state.title
if (stats.version > 0) title += `: ${stats.files} ${pluralize('file', stats.files)} (${pretty(stats.byteLength)})`
else if (stats.version === 0) title += ': (empty archive)'
if (state.opts.import) {
progressView = importUI(state) + '\n'
} else {
progressView = 'Not importing files.'
}
return output(`
${title}
${progressView}
${state.exiting ? exitMsg : chalk.dim('Ctrl+C to Exit')}
`)
}
================================================
FILE: src/ui/elements/key.js
================================================
var stringKey = require('dat-encoding').toStr
var chalk = require('chalk')
module.exports = function (key) {
return `${chalk.blue(`dat://${stringKey(key)}`)}`
}
================================================
FILE: src/ui/elements/pluralize.js
================================================
module.exports = function pluralize (str, val) {
return `${str}${val === 1 ? '' : 's'}`
}
================================================
FILE: src/ui/elements/version.js
================================================
var chalk = require('chalk')
module.exports = function (version) {
return `${chalk.green(`dat v${version}`)}`
}
================================================
FILE: src/ui/status.js
================================================
var output = require('neat-log/output')
var stringKey = require('dat-encoding').toStr
var pretty = require('prettier-bytes')
var chalk = require('chalk')
module.exports = statusUI
function statusUI (state) {
if (!state.dat) return 'Starting Dat program...'
var dat = state.dat
var stats = dat.stats.get()
return output(`
${chalk.blue('dat://' + stringKey(dat.key))}
${stats.files} files (${pretty(stats.byteLength)})
Version: ${chalk.bold(stats.version)}
`)
}
================================================
FILE: src/usage.js
================================================
module.exports = function (opts, help, usage) {
if (opts.version) {
var pkg = require('../package.json')
console.error(pkg.version)
process.exit(1)
}
var msg = `
dat <link> [<dir>] clone or sync link to <dir>
dat <dir> create and sync dat in directory
Other commands:
dat create <dir> create empty dat and dat.json
dat sync <dir> live sync files with the network
dat clone <link> [<dir>] download a dat via link to <dir>
dat pull update dat & exit
dat log log history for a dat
dat status get key & info about a local dat
dat keys [import/export] import and export private keys
Troubleshooting & Help:
dat help print this usage guide
dat <command> --help, -h print help for a specific command
dat --version, -v print the dat version
`
console.error(msg)
if (usage) {
console.error('General Options:')
console.error(usage)
}
console.error('Have fun using Dat! Learn more at docs.datproject.org')
process.exit(0)
}
================================================
FILE: test/auth.js
================================================
var test = require('tape')
var path = require('path')
var fs = require('fs')
var rimraf = require('rimraf')
var mkdirp = require('mkdirp')
var spawn = require('./helpers/spawn')
var help = require('./helpers')
var authServer = require('./helpers/auth-server')
var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js'))
var baseTestDir = help.testFolder()
var fixtures = path.join(__dirname, 'fixtures')
var port = process.env.PORT || 3000
var SERVER = 'http://localhost:' + port
var config = path.join(__dirname, '.datrc-test')
var opts = ' --server=' + SERVER + ' --config=' + config
dat += opts
rimraf.sync(config)
authServer(port, function (err, server, closeServer) {
if (err) throw err
if (!server) return
test('auth - whoami works when not logged in', function (t) {
var cmd = dat + ' whoami '
var st = spawn(t, cmd, { cwd: baseTestDir })
st.stderr.match(function (output) {
t.same(output.trim(), 'Not logged in.', 'printed correct output')
return true
})
st.stdout.empty()
st.end()
})
test('auth - register works', function (t) {
var cmd = dat + ' register --email=hello@bob.com --password=joe --username=joe'
var st = spawn(t, cmd, { cwd: baseTestDir })
st.stdout.match(function (output) {
t.same(output.trim(), 'Registered successfully.', 'output success message')
return true
})
st.stderr.empty()
st.end()
})
test('auth - login works', function (t) {
var cmd = dat + ' login --email=hello@bob.com --password=joe'
var st = spawn(t, cmd, { cwd: baseTestDir })
st.stdout.match(function (output) {
t.same(output.trim(), 'Logged in successfully.', 'output success message')
return true
})
st.stderr.empty()
st.end()
})
test('auth - whoami works', function (t) {
var cmd = dat + ' whoami'
var st = spawn(t, cmd, { cwd: baseTestDir })
st.stdout.match(function (output) {
t.same('hello@bob.com', output.trim(), 'email printed')
return true
})
st.stderr.empty()
st.end()
})
test('auth - publish before create fails', function (t) {
var cmd = dat + ' publish'
rimraf.sync(path.join(fixtures, '.dat'))
var st = spawn(t, cmd, { cwd: fixtures })
st.stdout.empty()
st.stderr.match(function (output) {
t.ok(output.indexOf('existing') > -1, 'Create archive before pub')
return true
})
st.end()
})
test('auth - create dat to publish', function (t) {
rimraf.sync(path.join(fixtures, '.dat'))
rimraf.sync(path.join(fixtures, 'dat.json'))
var cmd = dat + ' create --no-import'
var st = spawn(t, cmd, { cwd: fixtures })
st.stdout.match(function (output) {
var link = help.matchLink(output)
if (!link) return false
t.ok(link, 'prints link')
return true
})
st.stderr.empty()
st.end()
})
test('auth - publish our awesome dat', function (t) {
var cmd = dat + ' publish --name awesome'
var st = spawn(t, cmd, { cwd: fixtures })
st.stdout.match(function (output) {
var published = output.indexOf('Successfully published') > -1
if (!published) return false
t.ok(published, 'published')
return true
})
st.stderr.empty()
st.end()
})
test('auth - publish our awesome dat with bad dat.json url', function (t) {
fs.readFile(path.join(fixtures, 'dat.json'), function (err, contents) {
t.ifError(err)
var info = JSON.parse(contents)
var oldUrl = info.url
info.url = info.url.replace('e', 'a')
fs.writeFile(path.join(fixtures, 'dat.json'), JSON.stringify(info), function (err) {
t.ifError(err, 'error after write')
var cmd = dat + ' publish --name awesome'
var st = spawn(t, cmd, { cwd: fixtures })
st.stdout.match(function (output) {
var published = output.indexOf('Successfully published') > -1
if (!published) return false
t.ok(published, 'published')
t.same(help.datJson(fixtures).url, oldUrl, 'has dat.json with url')
return true
})
st.stderr.empty()
st.end()
})
})
})
test('auth - clone from registry', function (t) {
// MAKE SURE THESE MATCH WHAT is published above
// TODO: be less lazy and make a publish helper
var shortName = 'localhost:' + port + '/joe/awesome' // they'll never guess who wrote these tests
var baseDir = path.join(baseTestDir, 'dat_registry_dir')
mkdirp.sync(baseDir)
var downloadDir = path.join(baseDir, shortName.split('/').pop())
var cmd = dat + ' clone ' + shortName
var st = spawn(t, cmd, { cwd: baseDir })
st.stdout.match(function (output) {
var lookingFor = output.indexOf('Looking for') > -1
if (!lookingFor) return false
t.ok(lookingFor, 'starts looking for peers')
t.ok(output.indexOf(downloadDir) > -1, 'prints dir')
st.kill()
return true
})
st.stderr.empty()
st.end(function () {
rimraf.sync(downloadDir)
})
})
test('auth - publish our awesome dat without a dat.json file', function (t) {
rimraf(path.join(fixtures, 'dat.json'), function (err) {
t.ifError(err)
var cmd = dat + ' publish --name another-awesome'
var st = spawn(t, cmd, { cwd: fixtures })
st.stdout.match(function (output) {
var published = output.indexOf('Successfully published') > -1
if (!published) return false
t.ok(published, 'published')
t.same(help.datJson(fixtures).name, 'another-awesome', 'has dat.json with name')
return true
})
st.stderr.empty()
st.end(function () {
rimraf.sync(path.join(fixtures, '.dat'))
})
})
})
test('auth - bad clone from registry', function (t) {
var shortName = 'localhost:' + port + '/joe/not-at-all-awesome'
var baseDir = path.join(baseTestDir, 'dat_registry_dir_too')
mkdirp.sync(baseDir)
var downloadDir = path.join(baseDir, shortName.split('/').pop())
var cmd = dat + ' clone ' + shortName
var st = spawn(t, cmd, { cwd: baseDir })
st.stderr.match(function (output) {
t.same(output.trim(), 'Dat with that name not found.', 'not found')
st.kill()
return true
})
st.stdout.empty()
st.end(function () {
rimraf.sync(downloadDir)
})
})
test('auth - logout works', function (t) {
var cmd = dat + ' logout'
var st = spawn(t, cmd, { cwd: baseTestDir })
st.stdout.match(function (output) {
t.same('Logged out.', output.trim(), 'output correct')
return true
})
st.stderr.empty()
st.end()
})
test('auth - logout prints correctly when trying to log out twice', function (t) {
var cmd = dat + ' logout'
var st = spawn(t, cmd, { cwd: baseTestDir })
st.stderr.match(function (output) {
t.same('Not logged in.', output.trim(), 'output correct')
return true
})
st.stdout.empty()
st.end()
})
test('auth - whoami works after logging out', function (t) {
var cmd = dat + ' whoami'
var st = spawn(t, cmd, { cwd: baseTestDir })
st.stderr.match(function (output) {
t.same('Not logged in.', output.trim())
return true
})
st.stdout.empty()
st.end()
})
test.onFinish(function () {
closeServer(function () {
fs.unlink(config, function () {
// done!
})
})
})
})
================================================
FILE: test/clone.js
================================================
var fs = require('fs')
var path = require('path')
var test = require('tape')
var tempDir = require('temporary-directory')
var spawn = require('./helpers/spawn.js')
var help = require('./helpers')
var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js'))
test('clone - default opts', function (t) {
help.shareFixtures(function (_, shareDat) {
var key = shareDat.key.toString('hex')
tempDir(function (_, dir, cleanup) {
var cmd = dat + ' clone ' + key
var st = spawn(t, cmd, { cwd: dir })
var datDir = path.join(dir, key)
st.stdout.match(function (output) {
var downloadFinished = output.indexOf('Exiting') > -1
if (!downloadFinished) return false
var stats = shareDat.stats.get()
var fileRe = new RegExp(stats.files + ' files')
var bytesRe = new RegExp(/1\.\d KB/)
t.ok(output.match(fileRe), 'total size: files okay')
t.ok(output.match(bytesRe), 'total size: bytes okay')
t.ok(help.isDir(datDir), 'creates download directory')
var fileList = help.fileList(datDir).join(' ')
var hasCsvFile = fileList.indexOf('all_hour.csv') > -1
t.ok(hasCsvFile, 'csv file downloaded')
var hasDatFolder = fileList.indexOf('.dat') > -1
t.ok(hasDatFolder, '.dat folder created')
var hasSubDir = fileList.indexOf('folder') > -1
t.ok(hasSubDir, 'folder created')
var hasNestedDir = fileList.indexOf('nested') > -1
t.ok(hasNestedDir, 'nested folder created')
var hasHelloFile = fileList.indexOf('hello.txt') > -1
t.ok(hasHelloFile, 'hello.txt file downloaded')
st.kill()
return true
})
st.succeeds('exits after finishing download')
st.stderr.empty()
st.end(function () {
cleanup()
shareDat.close()
})
})
})
})
// Right now we aren't forcing this
// test('clone - errors on existing dir', function (t) {
// tempDir(function (_, dir, cleanup) {
// // make empty dat in directory
// Dat(dir, function (err, shareDat) {
// t.error(err, 'no error')
// // Try to clone to same dir
// shareDat.close(function () {
// var cmd = dat + ' clone ' + shareDat.key.toString('hex') + ' ' + dir
// var st = spawn(t, cmd)
// st.stdout.empty()
// st.stderr.match(function (output) {
// t.same(output.trim(), 'Existing archive in this directory. Use pull or sync to update.', 'Existing archive.')
// st.kill()
// return true
// })
// st.end(cleanup)
// })
// })
// })
// })
test('clone - specify dir', function (t) {
help.shareFixtures(function (_, shareDat) {
tempDir(function (_, dir, cleanup) {
var key = shareDat.key.toString('hex')
var customDir = 'my_dir'
var cmd = dat + ' clone ' + key + ' ' + customDir
var st = spawn(t, cmd, { cwd: dir })
st.stdout.match(function (output) {
var downloadFinished = output.indexOf('Exiting') > -1
if (!downloadFinished) return false
t.ok(help.isDir(path.join(dir, customDir)), 'creates download directory')
st.kill()
return true
})
st.succeeds('exits after finishing download')
st.stderr.empty()
st.end(function () {
cleanup()
shareDat.close()
})
})
})
})
test('clone - dat:// link', function (t) {
help.shareFixtures(function (_, shareDat) {
tempDir(function (_, dir, cleanup) {
var key = 'dat://' + shareDat.key.toString('hex') + '/'
var cmd = dat + ' clone ' + key + ' '
var downloadDir = path.join(dir, shareDat.key.toString('hex'))
var st = spawn(t, cmd, { cwd: dir })
st.stdout.match(function (output) {
var downloadFinished = output.indexOf('Exiting') > -1
if (!downloadFinished) return false
t.ok(help.isDir(path.join(downloadDir)), 'creates download directory')
st.kill()
return true
})
st.succeeds('exits after finishing download')
st.stderr.empty()
st.end(function () {
cleanup()
shareDat.close()
})
})
})
})
test('clone - datproject.org/key link', function (t) {
help.shareFixtures(function (_, shareDat) {
tempDir(function (_, dir, cleanup) {
var key = 'datproject.org/' + shareDat.key.toString('hex') + '/'
var cmd = dat + ' clone ' + key + ' '
var downloadDir = path.join(dir, shareDat.key.toString('hex'))
var st = spawn(t, cmd, { cwd: dir })
st.stdout.match(function (output) {
var downloadFinished = output.indexOf('Exiting') > -1
if (!downloadFinished) return false
t.ok(help.isDir(path.join(downloadDir)), 'creates download directory')
st.kill()
return true
})
st.succeeds('exits after finishing download')
st.stderr.empty()
st.end(function () {
cleanup()
shareDat.close()
})
})
})
})
// TODO: fix --temp for clones
// test('clone - with --temp', function (t) {
// // cmd: dat clone <link>
// help.shareFixtures(function (_, fixturesDat) {
// shareDat = fixturesDat
// var key = shareDat.key.toString('hex')
// var cmd = dat + ' clone ' + key + ' --temp'
// var st = spawn(t, cmd, {cwd: baseTestDir})
// var datDir = path.join(baseTestDir, key)
// st.stdout.match(function (output) {
// var downloadFinished = output.indexOf('Download Finished') > -1
// if (!downloadFinished) return false
// var stats = shareDat.stats.get()
// var fileRe = new RegExp(stats.filesTotal + ' files')
// var bytesRe = new RegExp(/1\.\d{1,2} kB/)
// t.ok(help.matchLink(output), 'prints link')
// t.ok(output.indexOf('dat-download-folder/' + key) > -1, 'prints dir')
// t.ok(output.match(fileRe), 'total size: files okay')
// t.ok(output.match(bytesRe), 'total size: bytes okay')
// t.ok(help.isDir(datDir), 'creates download directory')
// var fileList = help.fileList(datDir).join(' ')
// var hasCsvFile = fileList.indexOf('all_hour.csv') > -1
// t.ok(hasCsvFile, 'csv file downloaded')
// var hasDatFolder = fileList.indexOf('.dat') > -1
// t.ok(!hasDatFolder, '.dat folder not created')
// var hasSubDir = fileList.indexOf('folder') > -1
// t.ok(hasSubDir, 'folder created')
// var hasNestedDir = fileList.indexOf('nested') > -1
// t.ok(hasNestedDir, 'nested folder created')
// var hasHelloFile = fileList.indexOf('hello.txt') > -1
// t.ok(hasHelloFile, 'hello.txt file downloaded')
// st.kill()
// return true
// })
// st.succeeds('exits after finishing download')
// st.stderr.empty()
// st.end()
// })
// })
test('clone - invalid link', function (t) {
var key = 'best-key-ever'
var cmd = dat + ' clone ' + key
tempDir(function (_, dir, cleanup) {
var st = spawn(t, cmd, { cwd: dir })
var datDir = path.join(dir, key)
st.stderr.match(function (output) {
var error = output.indexOf('Could not resolve link') > -1
if (!error) return false
t.ok(error, 'has error')
t.ok(!help.isDir(datDir), 'download dir removed')
st.kill()
return true
})
st.end(cleanup)
})
})
test('clone - shortcut/stateless clone', function (t) {
help.shareFixtures(function (_, shareDat) {
var key = shareDat.key.toString('hex')
tempDir(function (_, dir, cleanup) {
var datDir = path.join(dir, key)
var cmd = dat + ' ' + key + ' ' + datDir + ' --exit'
var st = spawn(t, cmd)
st.stdout.match(function (output) {
var downloadFinished = output.indexOf('Exiting') > -1
if (!downloadFinished) return false
t.ok(help.isDir(datDir), 'creates download directory')
var fileList = help.fileList(datDir).join(' ')
var hasCsvFile = fileList.indexOf('all_hour.csv') > -1
t.ok(hasCsvFile, 'csv file downloaded')
var hasDatFolder = fileList.indexOf('.dat') > -1
t.ok(hasDatFolder, '.dat folder created')
var hasSubDir = fileList.indexOf('folder') > -1
t.ok(hasSubDir, 'folder created')
var hasNestedDir = fileList.indexOf('nested') > -1
t.ok(hasNestedDir, 'nested folder created')
var hasHelloFile = fileList.indexOf('hello.txt') > -1
t.ok(hasHelloFile, 'hello.txt file downloaded')
st.kill()
return true
})
st.succeeds('exits after finishing download')
st.stderr.empty()
st.end(function () {
cleanup()
shareDat.close()
})
})
})
})
// TODO: fix this
// test('clone - hypercore link', function (t) {
// help.shareFeed(function (_, key, close) {
// tempDir(function (_, dir, cleanup) {
// var cmd = dat + ' clone ' + key
// var st = spawn(t, cmd, {cwd: dir})
// var datDir = path.join(dir, key)
// st.stderr.match(function (output) {
// var error = output.indexOf('not a Dat Archive') > -1
// if (!error) return false
// t.ok(error, 'has error')
// t.ok(!help.isDir(datDir), 'download dir removed')
// st.kill()
// return true
// })
// st.end(function () {
// cleanup()
// close()
// })
// })
// })
// })
test('clone - specify directory containing dat.json', function (t) {
help.shareFixtures(function (_, shareDat) {
tempDir(function (_, dir, cleanup) {
fs.writeFileSync(path.join(dir, 'dat.json'), JSON.stringify({ url: shareDat.key.toString('hex') }), 'utf8')
// dat clone /dir
var cmd = dat + ' clone ' + dir
var st = spawn(t, cmd)
var datDir = dir
st.stdout.match(function (output) {
var downloadFinished = output.indexOf('Exiting') > -1
if (!downloadFinished) return false
var fileList = help.fileList(datDir).join(' ')
var hasCsvFile = fileList.indexOf('all_hour.csv') > -1
t.ok(hasCsvFile, 'csv file downloaded')
var hasDatFolder = fileList.indexOf('.dat') > -1
t.ok(hasDatFolder, '.dat folder created')
var hasSubDir = fileList.indexOf('folder') > -1
t.ok(hasSubDir, 'folder created')
var hasNestedDir = fileList.indexOf('nested') > -1
t.ok(hasNestedDir, 'nested folder created')
var hasHelloFile = fileList.indexOf('hello.txt') > -1
t.ok(hasHelloFile, 'hello.txt file downloaded')
st.kill()
return true
})
st.succeeds('exits after finishing download')
st.stderr.empty()
st.end(function () {
cleanup()
shareDat.close()
})
})
})
})
test('clone - specify directory containing dat.json with cwd', function (t) {
help.shareFixtures(function (_, shareDat) {
tempDir(function (_, dir, cleanup) {
fs.writeFileSync(path.join(dir, 'dat.json'), JSON.stringify({ url: shareDat.key.toString('hex') }), 'utf8')
// cd dir && dat clone /dir/dat.json
var cmd = dat + ' clone ' + dir
var st = spawn(t, cmd, { cwd: dir })
var datDir = dir
st.stdout.match(function (output) {
var downloadFinished = output.indexOf('Exiting') > -1
if (!downloadFinished) return false
var fileList = help.fileList(datDir).join(' ')
var hasCsvFile = fileList.indexOf('all_hour.csv') > -1
t.ok(hasCsvFile, 'csv file downloaded')
var hasDatFolder = fileList.indexOf('.dat') > -1
t.ok(hasDatFolder, '.dat folder created')
var hasSubDir = fileList.indexOf('folder') > -1
t.ok(hasSubDir, 'folder created')
var hasNestedDir = fileList.indexOf('nested') > -1
t.ok(hasNestedDir, 'nested folder created')
var hasHelloFile = fileList.indexOf('hello.txt') > -1
t.ok(hasHelloFile, 'hello.txt file downloaded')
st.kill()
return true
})
st.succeeds('exits after finishing download')
st.stderr.empty()
st.end(function () {
cleanup()
shareDat.close()
})
})
})
})
test('clone - specify dat.json path', function (t) {
help.shareFixtures(function (_, shareDat) {
tempDir(function (_, dir, cleanup) {
var datJsonPath = path.join(dir, 'dat.json')
fs.writeFileSync(datJsonPath, JSON.stringify({ url: shareDat.key.toString('hex') }), 'utf8')
// dat clone /dir/dat.json
var cmd = dat + ' clone ' + datJsonPath
var st = spawn(t, cmd)
var datDir = dir
st.stdout.match(function (output) {
var downloadFinished = output.indexOf('Exiting') > -1
if (!downloadFinished) return false
var fileList = help.fileList(datDir).join(' ')
var hasCsvFile = fileList.indexOf('all_hour.csv') > -1
t.ok(hasCsvFile, 'csv file downloaded')
var hasDatFolder = fileList.indexOf('.dat') > -1
t.ok(hasDatFolder, '.dat folder created')
var hasSubDir = fileList.indexOf('folder') > -1
t.ok(hasSubDir, 'folder created')
var hasNestedDir = fileList.indexOf('nested') > -1
t.ok(hasNestedDir, 'nested folder created')
var hasHelloFile = fileList.indexOf('hello.txt') > -1
t.ok(hasHelloFile, 'hello.txt file downloaded')
st.kill()
return true
})
st.succeeds('exits after finishing download')
st.stderr.empty()
st.end(function () {
cleanup()
shareDat.close()
})
})
})
})
test('clone - specify dat.json path with cwd', function (t) {
help.shareFixtures(function (_, shareDat) {
tempDir(function (_, dir, cleanup) {
var datJsonPath = path.join(dir, 'dat.json')
fs.writeFileSync(datJsonPath, JSON.stringify({ url: shareDat.key.toString('hex') }), 'utf8')
// cd /dir && dat clone /dir/dat.json
var cmd = dat + ' clone ' + datJsonPath
var st = spawn(t, cmd, { cwd: dir })
var datDir = dir
st.stdout.match(function (output) {
var downloadFinished = output.indexOf('Exiting') > -1
if (!downloadFinished) return false
var fileList = help.fileList(datDir).join(' ')
var hasCsvFile = fileList.indexOf('all_hour.csv') > -1
t.ok(hasCsvFile, 'csv file downloaded')
var hasDatFolder = fileList.indexOf('.dat') > -1
t.ok(hasDatFolder, '.dat folder created')
var hasSubDir = fileList.indexOf('folder') > -1
t.ok(hasSubDir, 'folder created')
var hasNestedDir = fileList.indexOf('nested') > -1
t.ok(hasNestedDir, 'nested folder created')
var hasHelloFile = fileList.indexOf('hello.txt') > -1
t.ok(hasHelloFile, 'hello.txt file downloaded')
st.kill()
return true
})
st.succeeds('exits after finishing download')
st.stderr.empty()
st.end(function () {
cleanup()
shareDat.close()
})
})
})
})
test('clone - specify dat.json + directory', function (t) {
help.shareFixtures(function (_, shareDat) {
tempDir(function (_, dir, cleanup) {
var datDir = path.join(dir, 'clone-dest')
var datJsonPath = path.join(dir, 'dat.json') // make dat.json in different dir
fs.mkdirSync(datDir)
fs.writeFileSync(datJsonPath, JSON.stringify({ url: shareDat.key.toString('hex') }), 'utf8')
// dat clone /dir/dat.json /dir/clone-dest
var cmd = dat + ' clone ' + datJsonPath + ' ' + datDir
var st = spawn(t, cmd)
st.stdout.match(function (output) {
var downloadFinished = output.indexOf('Exiting') > -1
if (!downloadFinished) return false
var fileList = help.fileList(datDir).join(' ')
var hasCsvFile = fileList.indexOf('all_hour.csv') > -1
t.ok(hasCsvFile, 'csv file downloaded')
var hasDatFolder = fileList.indexOf('.dat') > -1
t.ok(hasDatFolder, '.dat folder created')
var hasSubDir = fileList.indexOf('folder') > -1
t.ok(hasSubDir, 'folder created')
var hasNestedDir = fileList.indexOf('nested') > -1
t.ok(hasNestedDir, 'nested folder created')
var hasHelloFile = fileList.indexOf('hello.txt') > -1
t.ok(hasHelloFile, 'hello.txt file downloaded')
st.kill()
return true
})
st.succeeds('exits after finishing download')
st.stderr.empty()
st.end(function () {
cleanup()
shareDat.close()
})
})
})
})
================================================
FILE: test/create.js
================================================
var fs = require('fs')
var path = require('path')
var test = require('tape')
var tempDir = require('temporary-directory')
var Dat = require('dat-node')
var spawn = require('./helpers/spawn.js')
var help = require('./helpers')
var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js'))
var fixtures = path.join(__dirname, 'fixtures')
// os x adds this if you view the fixtures in finder and breaks the file count assertions
try { fs.unlinkSync(path.join(fixtures, '.DS_Store')) } catch (e) { /* ignore error */ }
// start without dat.json
try { fs.unlinkSync(path.join(fixtures, 'dat.json')) } catch (e) { /* ignore error */ }
test('create - default opts no import', function (t) {
tempDir(function (_, dir, cleanup) {
var cmd = dat + ' create --title data --description thing'
var st = spawn(t, cmd, { cwd: dir })
st.stdout.match(function (output) {
var datCreated = output.indexOf('Created empty Dat') > -1
if (!datCreated) return false
t.ok(help.isDir(path.join(dir, '.dat')), 'creates dat directory')
st.kill()
return true
})
st.succeeds('exits after create finishes')
st.stderr.empty()
st.end(cleanup)
})
})
test('create - errors on existing archive', function (t) {
tempDir(function (_, dir, cleanup) {
Dat(dir, function (err, dat) {
t.error(err, 'no error')
dat.close(function () {
var cmd = dat + ' create --title data --description thing'
var st = spawn(t, cmd, { cwd: dir })
st.stderr.match(function (output) {
t.ok(output, 'errors')
st.kill()
return true
})
st.end()
})
})
})
})
test('create - sync after create ok', function (t) {
tempDir(function (_, dir, cleanup) {
var cmd = dat + ' create --title data --description thing'
var st = spawn(t, cmd, { cwd: dir, end: false })
st.stdout.match(function (output) {
var connected = output.indexOf('Created empty Dat') > -1
if (!connected) return false
doSync()
return true
})
function doSync () {
var cmd = dat + ' sync '
var st = spawn(t, cmd, { cwd: dir })
st.stdout.match(function (output) {
var connected = output.indexOf('Sharing') > -1
if (!connected) return false
st.kill()
return true
})
st.stderr.empty()
st.end(cleanup)
}
})
})
test('create - init alias', function (t) {
tempDir(function (_, dir, cleanup) {
var cmd = dat + ' init --title data --description thing'
var st = spawn(t, cmd, { cwd: dir })
st.stdout.match(function (output) {
var datCreated = output.indexOf('Created empty Dat') > -1
if (!datCreated) return false
t.ok(help.isDir(path.join(dir, '.dat')), 'creates dat directory')
st.kill()
return true
})
st.succeeds('exits after create finishes')
st.stderr.empty()
st.end(cleanup)
})
})
test('create - with path', function (t) {
tempDir(function (_, dir, cleanup) {
var cmd = dat + ' init ' + dir + ' --title data --description thing'
var st = spawn(t, cmd)
st.stdout.match(function (output) {
var datCreated = output.indexOf('Created empty Dat') > -1
if (!datCreated) return false
t.ok(help.isDir(path.join(dir, '.dat')), 'creates dat directory')
st.kill()
return true
})
st.succeeds('exits after create finishes')
st.stderr.empty()
st.end(cleanup)
})
})
================================================
FILE: test/dat-node.js
================================================
var test = require('tape')
var ram = require('random-access-memory')
var Dat = require('..')
test('dat-node: require dat-node + make a dat', function (t) {
Dat(ram, function (err, dat) {
t.error(err, 'no error')
t.ok(dat, 'makes dat')
t.pass('yay')
t.end()
})
})
================================================
FILE: test/doctor.js
================================================
// var path = require('path')
// var test = require('tape')
// var spawn = require('./helpers/spawn.js')
// var help = require('./helpers')
//
// TODO
// dat-doctor requires interactive testing right now...
// var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js'))
// test('misc - doctor option works ', function (t) {
// var st = spawn(t, dat + ' doctor', {end: false})
// st.stderr.match(function (output) {
// var readyPeer = output.indexOf('Waiting for incoming connections') > -1
// if (!readyPeer) return false
// if (!process.env.TRAVIS) {
// // Not working on v4/v7 travis but can't reproduce locally
// t.ok(output.indexOf('UTP') > -1, 'doctor connects to public peer via UTP')
// }
// t.ok(output.indexOf('TCP') > -1, 'doctor connects to public peer via TCP')
// var key = help.matchLink(output)
// startPhysiciansAssistant(key)
// return true
// }, 'doctor started')
// function startPhysiciansAssistant (link) {
// var assist = spawn(t, dat + ' doctor ' + link, {end: false})
// assist.stderr.match(function (output) {
// var readyPeer = output.indexOf('Waiting for incoming connections') > -1
// if (!readyPeer) return false
// t.same(help.matchLink(output), link, 'key of peer matches')
// t.ok(readyPeer, 'starts looking for peers')
// t.skip(output.indexOf('Remote peer echoed expected data back') > -1, 'echo data back')
// st.kill()
// return true
// })
// assist.end(function () {
// t.end()
// })
// }
// })
================================================
FILE: test/fixtures/all_hour.csv
================================================
time,latitude,longitude,depth,mag,magType,nst,gap,dmin,rms,net,id,updated,place,type
2014-04-30T03:34:57.000Z,60.0366,-141.2214,14.6,1.4,ml,,,,1.51,ak,ak11246293,2014-04-30T03:39:27.956Z,"67km E of Cape Yakataga, Alaska",earthquake
2014-04-30T03:16:54.860Z,33.9233322,-117.9376678,0.81,2.4,ml,71,51,0.02487,0.35,ci,ci37218696,2014-04-30T03:35:24.239Z,"1km SE of La Habra, California",earthquake
2014-04-30T03:03:09.000Z,61.126,-149.7035,28.2,1,ml,,,,0.75,ak,ak11246291,2014-04-30T03:09:51.716Z,"14km SE of Anchorage, Alaska",earthquake
2014-04-30T02:57:51.800Z,37.3798,-122.1912,4.5,1.3,Md,8,104.4,0.01796631,0.02,nc,nc72212216,2014-04-30T03:28:05.271Z,"2km SSE of Ladera, California",earthquake
2014-04-30T02:51:26.000Z,60.2062,-147.7298,0,1.8,ml,,,,0.1,ak,ak11246289,2014-04-30T02:54:55.916Z,"82km SE of Whittier, Alaska",earthquake
2014-04-30T02:48:54.000Z,60.4899,-149.4879,28.5,1.4,ml,,,,0.54,ak,ak11246287,2014-04-30T02:54:51.149Z,"36km N of Bear Creek, Alaska",earthquake
2014-04-30T02:47:44.100Z,38.819,-122.7952,3.3,0.8,Md,12,75.6,0.00898315,0.02,nc,nc72212211,2014-04-30T03:17:09.173Z,"5km NW of The Geysers, California",earthquake
2014-04-30T02:46:11.100Z,37.9812,-122.054,14.7,2.1,Md,35,126,0.09881468,0.09,nc,nc72212206,2014-04-30T03:38:06.391Z,"1km E of Pacheco, California",earthquake
2014-04-30T02:44:49.000Z,61.3482,-151.3611,48.5,1.7,ml,,,,0.92,ak,ak11246285,2014-04-30T02:54:49.809Z,"73km N of Nikiski, Alaska",earthquake
================================================
FILE: test/fixtures/folder/nested/hello.txt
================================================
code for science and society
================================================
FILE: test/helpers/auth-server.js
================================================
var path = require('path')
var rimraf = require('rimraf')
var Server, initDb
try {
Server = require('dat-registry-api/server')
initDb = require('dat-registry-api/server/database/init')
} catch (e) {
console.log('Disabling auth tests, run `npm install dat-registry-api` to enable them.')
}
module.exports = createServer
function createServer (port, cb) {
if (!Server || !initDb) return cb(null)
var config = {
mixpanel: 'nothing',
email: {
fromEmail: 'hi@example.com'
},
township: {
secret: 'very secret code',
db: path.join(__dirname, '..', 'test-township.db')
},
db: {
dialect: 'sqlite3',
connection: { filename: path.join(__dirname, '..', 'test-sqlite.db') },
useNullAsDefault: true
},
archiver: path.join(__dirname, '..', 'test-archiver'),
whitelist: false,
port: port || 8888
}
rimraf.sync(config.archiver)
rimraf.sync(config.db.connection.filename)
rimraf.sync(config.township.db)
initDb(config.db, function (err, db) {
if (err) return cb(err)
const server = Server(config, db)
server.listen(config.port, function () {
console.log('listening', config.port)
})
cb(null, server, close)
function close (cb) {
server.close(function () {
rimraf.sync(config.township.db)
rimraf.sync(config.db.connection.filename)
process.exit()
})
}
})
}
================================================
FILE: test/helpers/index.js
================================================
var fs = require('fs')
var os = require('os')
var path = require('path')
var mkdirp = require('mkdirp')
var rimraf = require('rimraf')
var encoding = require('dat-encoding')
var recursiveReadSync = require('recursive-readdir-sync')
var Dat = require('dat-node')
var fetch = require('node-fetch')
module.exports.matchLink = matchDatLink
module.exports.isDir = isDir
module.exports.testFolder = newTestFolder
module.exports.datJson = datJson
module.exports.shareFixtures = shareFixtures
module.exports.fileList = fileList
module.exports.fetchText = fetchText
function shareFixtures (opts, cb) {
if (typeof opts === 'function') return shareFixtures(null, opts)
opts = opts || {}
var fixtures = path.join(__dirname, '..', 'fixtures')
// os x adds this if you view the fixtures in finder and breaks the file count assertions
try { fs.unlinkSync(path.join(fixtures, '.DS_Store')) } catch (e) { /* ignore error */ }
if (opts.resume !== true) rimraf.sync(path.join(fixtures, '.dat'))
Dat(fixtures, {}, function (err, dat) {
if (err) throw err
dat.trackStats()
dat.joinNetwork()
if (opts.import === false) return cb(null, dat)
dat.importFiles({ watch: false }, function (err) {
if (err) throw err
cb(null, dat)
})
})
}
function fileList (dir) {
try {
return recursiveReadSync(dir)
} catch (e) {
return []
}
}
function newTestFolder () {
var tmpdir = path.join(os.tmpdir(), 'dat-download-folder')
rimraf.sync(tmpdir)
mkdirp.sync(tmpdir)
return tmpdir
}
function matchDatLink (str) {
var match = str.match(/[A-Za-z0-9]{64}/)
if (!match) return false
var key
try {
key = encoding.toStr(match[0].trim())
} catch (e) {
return false
}
return key
}
function datJson (filepath) {
try {
return JSON.parse(fs.readFileSync(path.join(filepath, 'dat.json')))
} catch (e) {
return {}
}
}
function isDir (dir) {
try {
return fs.statSync(dir).isDirectory()
} catch (e) {
return false
}
}
function fetchText (url) {
let resp = {}
return fetch(url).then(function (res) {
resp = res
return resp.text()
}).then(function (body) {
return { resp, body, error: null }
}).catch(function (error) {
return { error, resp: {}, body: null }
})
}
================================================
FILE: test/helpers/spawn.js
================================================
var spawn = require('tape-spawn')
var fs = require('fs')
// happens once at require time
// https://github.com/AndreasMadsen/execspawn/issues/2
var hasBash = fs.existsSync('/bin/bash')
module.exports = function (t, cmd, opts) {
opts = opts || {}
if (hasBash) opts.shell = '/bin/bash' // override default of /bin/sh
return spawn(t, cmd, opts)
}
================================================
FILE: test/http.js
================================================
var path = require('path')
var test = require('tape')
var rimraf = require('rimraf')
var spawn = require('./helpers/spawn.js')
var { fetchText } = require('./helpers')
var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js'))
var fixtures = path.join(__dirname, 'fixtures')
test('http - share with http', function (t) {
rimraf.sync(path.join(fixtures, '.dat'))
var cmd = dat + ' share --http'
var st = spawn(t, cmd, { cwd: fixtures })
st.stdout.match(function (output) {
var sharingHttp = output.indexOf('Serving files over http') > -1
if (!sharingHttp) return false
fetchText('http://localhost:8080').then(function ({ resp, body, error }) {
t.error(error, 'no error')
t.ok(resp.status === 200, 'okay status')
t.ok(body)
fetchText('http://localhost:8080/folder/nested/hello.txt').then(function ({ resp, body, error }) {
t.error(error, 'no error')
t.ok(resp.status === 200, 'okay status')
t.same(body, 'code for science and society', 'body of file okay')
st.kill()
})
})
return true
})
st.stderr.empty()
st.end()
})
test('http - share with http other port', function (t) {
rimraf.sync(path.join(fixtures, '.dat'))
var cmd = dat + ' share --http 3333'
var st = spawn(t, cmd, { cwd: fixtures })
st.stdout.match(function (output) {
var sharingHttp = output.indexOf('Serving files over http') > -1
if (!sharingHttp) return false
fetchText('http://localhost:3333').then(function ({ resp, body, error }) {
t.error(error, 'no error')
t.ok(resp.status === 200, 'okay status')
t.ok(body)
fetchText('http://localhost:3333/folder/nested/hello.txt').then(function ({ resp, body, error }) {
t.error(error, 'no error')
t.ok(resp.status === 200, 'okay status')
t.same(body, 'code for science and society', 'body of file okay')
st.kill()
})
})
return true
})
st.stderr.empty()
st.end()
})
================================================
FILE: test/keys.js
================================================
var fs = require('fs')
var path = require('path')
var test = require('tape')
var rimraf = require('rimraf')
var tempDir = require('temporary-directory')
var spawn = require('./helpers/spawn.js')
var help = require('./helpers')
var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js'))
var fixtures = path.join(__dirname, 'fixtures')
test('keys - print keys', function (t) {
help.shareFixtures(function (_, shareDat) {
shareDat.close(function () {
var cmd = dat + ' keys '
var st = spawn(t, cmd, { cwd: fixtures })
st.stdout.match(function (output) {
if (output.indexOf('dat://') === -1) return false
t.ok(output.indexOf(shareDat.key.toString('hex') > -1), 'prints key')
st.kill()
return true
})
st.stderr.empty()
st.end()
})
})
})
test('keys - print discovery key', function (t) {
help.shareFixtures(function (_, shareDat) {
shareDat.close(function () {
var cmd = dat + ' keys --discovery'
var st = spawn(t, cmd, { cwd: fixtures })
st.stdout.match(function (output) {
if (output.indexOf('Discovery') === -1) return false
t.ok(output.indexOf(shareDat.key.toString('hex') > -1), 'prints key')
t.ok(output.indexOf(shareDat.archive.discoveryKey.toString('hex') > -1), 'prints discovery key')
st.kill()
return true
})
st.stderr.empty()
st.end()
})
})
})
if (!process.env.TRAVIS) {
test('keys - export & import secret key', function (t) {
help.shareFixtures(function (_, shareDat) {
var key = shareDat.key.toString('hex')
tempDir(function (_, dir, cleanup) {
var cmd = dat + ' clone ' + key
var st = spawn(t, cmd, { cwd: dir, end: false })
var datDir = path.join(dir, key)
st.stdout.match(function (output) {
var downloadFinished = output.indexOf('Exiting') > -1
if (!downloadFinished) return false
st.kill()
shareDat.close(exchangeKeys)
return true
})
st.stderr.empty()
function exchangeKeys () {
var secretKey = null
var exportKey = dat + ' keys export'
var st = spawn(t, exportKey, { cwd: fixtures, end: false })
st.stdout.match(function (output) {
if (!output) return false
secretKey = output.trim()
st.kill()
importKey()
return true
})
st.stderr.empty()
function importKey () {
var exportKey = dat + ' keys import'
var st = spawn(t, exportKey, { cwd: datDir })
st.stdout.match(function (output) {
if (!output.indexOf('secret key') === -1) return false
st.stdin.write(secretKey + '\r')
if (output.indexOf('Successful import') === -1) return false
t.ok(fs.statSync(path.join(datDir, '.dat', 'metadata.ogd')), 'original dat file exists')
st.kill()
return true
})
st.stderr.empty()
st.end(function () {
rimraf.sync(path.join(fixtures, '.dat'))
cleanup()
})
}
}
})
})
})
}
================================================
FILE: test/pull.js
================================================
var path = require('path')
var test = require('tape')
var tempDir = require('temporary-directory')
var spawn = require('./helpers/spawn.js')
var help = require('./helpers')
var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js'))
test('pull - errors without clone first', function (t) {
tempDir(function (_, dir, cleanup) {
var cmd = dat + ' pull'
var st = spawn(t, cmd, { cwd: dir })
st.stderr.match(function (output) {
t.ok('No existing archive', 'Error: no existing archive')
st.kill()
return true
})
st.end(cleanup)
})
})
test('pull - default opts', function (t) {
// import false so we can pull files later
help.shareFixtures({ import: false }, function (_, fixturesDat) {
tempDir(function (_, dir, cleanup) {
// clone initial dat
var cmd = dat + ' clone ' + fixturesDat.key.toString('hex') + ' ' + dir
var st = spawn(t, cmd, { end: false })
st.stdout.match(function (output) {
var synced = output.indexOf('dat synced') > -1
if (!synced) return false
st.kill()
fixturesDat.close(doPull)
return true
})
function doPull () {
// TODO: Finish this one. Need some bug fixes on empty pulls =(
help.shareFixtures({ resume: true, import: true }, function (_, fixturesDat) {
var cmd = dat + ' pull'
var st = spawn(t, cmd, { cwd: dir })
st.stdout.match(function (output) {
var downloadFinished = output.indexOf('dat sync') > -1
if (!downloadFinished) return false
st.kill()
return true
})
st.succeeds('exits after finishing download')
st.stderr.empty()
st.end(function () {
fixturesDat.close()
})
})
}
})
})
})
// test('pull - default opts', function (t) {
// // cmd: dat pull
// // import the files to the sharer so we can pull new data
// shareDat.importFiles(function (err) {
// if (err) throw err
// var datDir = path.join(baseTestDir, shareKey)
// var cmd = dat + ' pull'
// var st = spawn(t, cmd, {cwd: datDir})
// st.stdout.match(function (output) {
// var downloadFinished = output.indexOf('Download Finished') > -1
// if (!downloadFinished) return false
// var stats = shareDat.stats.get()
// var fileRe = new RegExp(stats.filesTotal + ' files')
// var bytesRe = new RegExp(/1\.\d{1,2} kB/)
// t.ok(help.matchLink(output), 'prints link')
// t.ok(output.indexOf('dat-download-folder/' + shareKey) > -1, 'prints dir')
// t.ok(output.match(fileRe), 'total size: files okay')
// t.ok(output.match(bytesRe), 'total size: bytes okay')
// t.ok(help.isDir(datDir), 'creates download directory')
// var fileList = help.fileList(datDir).join(' ')
// var hasCsvFile = fileList.indexOf('all_hour.csv') > -1
// t.ok(hasCsvFile, 'csv file downloaded')
// var hasDatFolder = fileList.indexOf('.dat') > -1
// t.ok(hasDatFolder, '.dat folder created')
// var hasSubDir = fileList.indexOf('folder') > -1
// t.ok(hasSubDir, 'folder created')
// var hasNestedDir = fileList.indexOf('nested') > -1
// t.ok(hasNestedDir, 'nested folder created')
// var hasHelloFile = fileList.indexOf('hello.txt') > -1
// t.ok(hasHelloFile, 'hello.txt file downloaded')
// st.kill()
// return true
// })
// st.succeeds('exits after finishing download')
// st.stderr.empty()
// st.end()
// })
// })
// test('pull - with dir arg', function (t) {
// var dirName = shareKey
// var datDir = path.join(baseTestDir, shareKey)
// var cmd = dat + ' pull ' + dirName
// var st = spawn(t, cmd, {cwd: baseTestDir})
// st.stdout.match(function (output) {
// var downloadFinished = output.indexOf('Download Finished') > -1
// if (!downloadFinished) return false
// t.ok(output.indexOf('dat-download-folder/' + dirName) > -1, 'prints dir')
// t.ok(help.isDir(datDir), 'creates download directory')
// st.kill()
// return true
// })
// st.succeeds('exits after finishing download')
// st.stderr.empty()
// st.end()
// })
================================================
FILE: test/share.js
================================================
// var fs = require('fs')
// var path = require('path')
// var test = require('tape')
// var rimraf = require('rimraf')
// var spawn = require('./helpers/spawn.js')
// var help = require('./helpers')
// var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js'))
// if (process.env.TRAVIS) dat += ' --no-watch '
// var fixtures = path.join(__dirname, 'fixtures')
// // os x adds this if you view the fixtures in finder and breaks the file count assertions
// try { fs.unlinkSync(path.join(fixtures, '.DS_Store')) } catch (e) { /* ignore error */ }
// // start without dat.json
// try { fs.unlinkSync(path.join(fixtures, 'dat.json')) } catch (e) { /* ignore error */ }
// test('share - default opts', function (t) {
// rimraf.sync(path.join(fixtures, '.dat'))
// var cmd = dat + ' share'
// var st = spawn(t, cmd, {cwd: fixtures})
// st.stdout.match(function (output) {
// var importFinished = output.indexOf('Total Size') > -1
// if (!importFinished) return false
// t.ok(help.isDir(path.join(fixtures, '.dat')), 'creates dat directory')
// t.ok(output.indexOf('Looking for connections') > -1, 'network')
// st.kill()
// return true
// })
// st.stderr.empty()
// st.end()
// })
// test('share - with dir arg', function (t) {
// rimraf.sync(path.join(fixtures, '.dat'))
// var cmd = dat + ' share ' + fixtures
// var st = spawn(t, cmd)
// st.stdout.match(function (output) {
// var importFinished = output.indexOf('Total Size') > -1
// if (!importFinished) return false
// t.ok(help.isDir(path.join(fixtures, '.dat')), 'creates dat directory')
// t.ok(output.indexOf('Looking for connections') > -1, 'network')
// st.kill()
// return true
// })
// st.stderr.empty()
// st.end()
// })
// test.onFinish(function () {
// rimraf.sync(path.join(fixtures, '.dat'))
// })
================================================
FILE: test/sync-owner.js
================================================
// var fs = require('fs')
// var net = require('net')
// var path = require('path')
// var test = require('tape')
// var mkdirp = require('mkdirp')
// var rimraf = require('rimraf')
// var Dat = require('dat-node')
// var spawn = require('./helpers/spawn.js')
// var help = require('./helpers')
// var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js'))
// if (process.env.TRAVIS) dat += ' --no-watch '
// var fixtures = path.join(__dirname, 'fixtures')
// var downDat
// // os x adds this if you view the fixtures in finder and breaks the file count assertions
// try { fs.unlinkSync(path.join(fixtures, '.DS_Store')) } catch (e) { /* ignore error */ }
// test('sync-owner - errors without create first', function (t) {
// rimraf.sync(path.join(fixtures, '.dat'))
// // cmd: dat sync
// var cmd = dat + ' sync'
// var st = spawn(t, cmd, {cwd: fixtures})
// st.stderr.match(function (output) {
// var hasError = output.indexOf('No existing archive') > -1
// t.ok(hasError, 'emits error')
// st.kill()
// return true
// })
// st.end()
// })
// test('sync-owner - create a dat for syncing', function (t) {
// rimraf.sync(path.join(fixtures, '.dat'))
// // cmd: dat create
// var cmd = dat + ' create --import'
// var st = spawn(t, cmd, {cwd: fixtures})
// st.stdout.match(function (output) {
// var importFinished = output.indexOf('import finished') > -1
// if (!importFinished) return false
// st.kill()
// return true
// })
// st.stderr.empty()
// st.end()
// })
// test('sync-owner - default opts', function (t) {
// // cmd: dat sync
// var cmd = dat + ' sync'
// var st = spawn(t, cmd, {cwd: fixtures, end: false})
// var key
// st.stdout.match(function (output) {
// var sharing = output.indexOf('Dat Network') > -1
// if (!sharing) return false
// key = help.matchLink(output)
// t.ok(key, 'prints link')
// t.ok(output.indexOf('tests/fixtures') > -1, 'prints dir')
// downloadDat()
// return true
// })
// st.stderr.empty()
// st.end()
// function downloadDat () {
// var downloadDir = path.join(help.testFolder(), '' + Date.now())
// mkdirp.sync(downloadDir)
// Dat(downloadDir, { key: key }, function (err, tmpDat) {
// if (err) throw err
// downDat = tmpDat
// downDat.joinNetwork()
// downDat.network.swarm.once('connection', function () {
// t.pass('downloader connects')
// downDat.close(function () {
// rimraf.sync(downDat.path)
// t.end()
// })
// })
// })
// }
// })
// test('sync-owner - create without import for syncing', function (t) {
// rimraf.sync(path.join(fixtures, '.dat'))
// // cmd: dat create
// var cmd = dat + ' create'
// var st = spawn(t, cmd, {cwd: fixtures})
// st.stdout.match(function (output) {
// if (output.indexOf('created') > -1) return true
// return false
// })
// st.succeeds()
// st.end()
// })
// test('sync-owner - imports after no-import create', function (t) {
// // cmd: dat sync
// var cmd = dat + ' sync'
// var st = spawn(t, cmd, {cwd: fixtures})
// st.stdout.match(function (output) {
// // have to check both for local test (watching) and travis (sharing)
// var sharing = output.indexOf('Watching') > -1 || output.indexOf('Sharing latest') > -1
// if (!sharing) return false
// var fileRe = new RegExp('2 files')
// var bytesRe = new RegExp(/1\.\d{1,2} kB/)
// t.ok(help.matchLink(output), 'prints link')
// t.ok(output.indexOf('tests/fixtures') > -1, 'prints dir')
// t.ok(output.match(fileRe), 'total size: files okay')
// t.ok(output.match(bytesRe), 'total size: bytes okay')
// st.kill()
// return true
// })
// st.stderr.empty()
// st.end()
// })
// // TODO: this test is causing serious memory issues.
// // HELP. Maybe related to https://github.com/datproject/dat-node/issues/71
// // test('sync-owner - turn off ignore hidden', function (t) {
// // // cmd: dat sync
// // var hiddenFile = path.join(fixtures, '.hidden-file')
// // var cmd = dat + ' sync --no-ignore-hidden'
// // fs.writeFile(hiddenFile, 'You cannot see me', function (err) {
// // t.error(err)
// // var st = spawn(t, cmd, {cwd: fixtures, end: false})
// // var key
// // st.stdout.match(function (output) {
// // var sharing = output.indexOf('Dat Network') > -1
// // if (!sharing) return false
// // key = help.matchLink(output)
// // downloadDat()
// // return true
// // })
// // st.stderr.empty()
// // st.end()
// // function downloadDat () {
// // var downloadDir = path.join(help.testFolder(), '' + Date.now())
// // mkdirp.sync(downloadDir)
// // Dat(downloadDir, { key: key }, function (err, downDat) {
// // if (err) throw err
// // downDat.joinNetwork()
// // downDat.network.swarm.once('connection', function () {
// // downDat.archive.list({live: false}, function (err, data) {
// // t.error(err)
// // var hasHiddenFile = data.filter(function (entry) {
// // return entry.name === '.hidden-file'
// // })
// // t.ok(hasHiddenFile.length, 'hidden file in archive')
// // downDat.network.swarm.close(function () {
// // process.nextTick(function () {
// // downDat.close(function () {
// // rimraf(downDat.path, function () {
// // fs.unlink(hiddenFile, function () {
// // t.end()
// // })
// // })
// // })
// // })
// // })
// // })
// // })
// // })
// // }
// // })
// // })
// test('sync-owner - port and utp options', function (t) {
// var port = 3281
// var cmd = dat + ' sync --port ' + port + ' --no-utp'
// var st = spawn(t, cmd, {cwd: fixtures, end: false})
// st.stderr.empty()
// var server = net.createServer()
// server.once('error', function (err) {
// if (err.code !== 'EADDRINUSE') return t.error(err)
// t.skip('TODO: correct port in use')
// done()
// })
// server.once('listening', function () {
// t.skip(`TODO: port ${server.address().port} should be in use`)
// done()
// })
// server.listen(port)
// t.skip('TODO: check utp option') // TODO: how to check utp?
// function done () {
// server.close(function () {
// st.kill()
// t.end()
// })
// }
// })
// test('sync-owner - shorthand', function (t) {
// var cmd = dat + ' .'
// var st = spawn(t, cmd, {cwd: fixtures})
// st.stdout.match(function (output) {
// var sharing = output.indexOf('Looking for connections') > -1
// if (!sharing) return false
// t.ok(help.matchLink(output), 'prints link')
// st.kill()
// return true
// })
// st.stderr.empty()
// st.end()
// })
// test('sync-owner - dir argument', function (t) {
// var cmd = dat + ' sync ' + fixtures
// var st = spawn(t, cmd)
// st.stdout.match(function (output) {
// var sharing = output.indexOf('Looking for connections') > -1
// if (!sharing) return false
// t.ok(help.matchLink(output), 'prints link')
// st.kill()
// return true
// })
// st.stderr.empty()
// st.end()
// })
// if (!process.env.TRAVIS) {
// test('sync-owner - live', function (t) {
// var liveFile = path.join(fixtures, 'live.txt')
// var wroteFile = false
// var cmd = dat + ' sync --watch'
// var st = spawn(t, cmd, {cwd: fixtures})
// st.stdout.match(function (output) {
// var watching = output.indexOf('Watching for file changes') > -1
// if (!watching) return false
// else if (!wroteFile) {
// fs.writeFileSync(liveFile, 'hello')
// wroteFile = true
// }
// var fileImported = output.indexOf('live.txt') > -1
// if (!fileImported) return false
// t.ok(fileImported, 'prints live file output')
// t.ok(output.indexOf('3 files') > -1, 'total size: files okay')
// fs.unlinkSync(liveFile)
// st.kill()
// return true
// })
// st.stderr.empty()
// st.end()
// })
// }
// test.onFinish(function () {
// rimraf.sync(path.join(fixtures, '.dat'))
// })
================================================
FILE: test/sync-remote.js
================================================
// var path = require('path')
// var test = require('tape')
// var rimraf = require('rimraf')
// var spawn = require('./helpers/spawn.js')
// var help = require('./helpers')
// var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js'))
// var baseTestDir = help.testFolder()
// var shareDat
// var syncDir
// test('sync-remote - default opts', function (t) {
// // cmd: dat sync
// var key
// help.shareFixtures({import: false}, function (_, fixturesDat) {
// shareDat = fixturesDat
// key = shareDat.key.toString('hex')
// syncDir = path.join(baseTestDir, key)
// makeClone(function () {
// shareDat.importFiles(function () {
// var cmd = dat + ' sync'
// var st = spawn(t, cmd, {cwd: syncDir})
// st.stdout.match(function (output) {
// var updated = output.indexOf('Files updated') > -1
// if (!updated) return false
// var fileRe = new RegExp('3 files')
// var bytesRe = new RegExp(/1\.\d{1,2} kB/)
// key = help.matchLink(output)
// t.ok(key, 'prints link')
// t.ok(output.indexOf('dat-download-folder/' + key) > -1, 'prints dir')
// t.ok(output.match(fileRe), 'total size: files okay')
// t.ok(output.match(bytesRe), 'total size: bytes okay')
// st.kill()
// return true
// })
// st.stderr.empty()
// st.end()
// })
// })
// })
// function makeClone (cb) {
// var cmd = dat + ' clone ' + key
// var st = spawn(t, cmd, {cwd: baseTestDir, end: false})
// st.stdout.match(function (output) {
// var downloadFinished = output.indexOf('Download Finished') > -1
// if (!downloadFinished) return false
// st.kill()
// cb()
// return true
// })
// st.stderr.empty()
// }
// })
// test('sync-remote - shorthand sync', function (t) {
// // cmd: dat sync
// var cmd = dat + ' .'
// var st = spawn(t, cmd, {cwd: syncDir})
// st.stdout.match(function (output) {
// var syncing = output.indexOf('Syncing Dat Archive') > -1
// if (!syncing) return false
// t.ok(help.matchLink(output), 'prints link')
// st.kill()
// return true
// })
// st.stderr.empty()
// st.end()
// })
// test('sync-remote - dir arg', function (t) {
// var cmd = dat + ' ' + syncDir
// var st = spawn(t, cmd)
// st.stdout.match(function (output) {
// var syncing = output.indexOf('Syncing Dat Archive') > -1
// if (!syncing) return false
// t.ok(help.matchLink(output), 'prints link')
// st.kill()
// return true
// })
// st.stderr.empty()
// st.end()
// })
// test('close sharer', function (t) {
// shareDat.close(function () {
// rimraf.sync(path.join(shareDat.path, '.dat'))
// t.end()
// })
// })
// test.onFinish(function () {
// rimraf.sync(baseTestDir)
// })
================================================
FILE: test/usage.js
================================================
var path = require('path')
var test = require('tape')
var spawn = require('./helpers/spawn.js')
var dat = path.resolve(path.join(__dirname, '..', 'bin', 'cli.js'))
var version = require('../package.json').version
test('usage - prints usage', function (t) {
var d = spawn(t, dat)
d.stderr.match(function (output) {
var usage = output.indexOf('dat <link> ') > -1
if (!usage) return false
return true
})
d.end()
})
test('usage - prints version', function (t) {
var d = spawn(t, dat + ' -v')
d.stderr.match(function (output) {
var ver = output.indexOf(version) > -1
if (!ver) return false
return true
})
d.end()
})
test('usage - also prints version', function (t) {
var d = spawn(t, dat + ' -v')
d.stderr.match(function (output) {
var ver = output.indexOf(version) > -1
if (!ver) return false
return true
})
d.end()
})
test('usage - help prints usage', function (t) {
var d = spawn(t, dat + ' help')
d.stderr.match(function (output) {
var usage = output.indexOf('dat <link> ') > -1
if (!usage) return false
return true
})
d.end()
})
gitextract_aa8sr91t/
├── .github/
│ ├── FUNDING.yml
│ └── issue_template.md
├── .gitignore
├── .travis.yml
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── LICENSE
├── README.md
├── appveyor.yml
├── bin/
│ └── cli.js
├── changelog.md
├── download.sh
├── index.js
├── package.json
├── package.sh
├── scripts/
│ └── auth-server.js
├── snap/
│ └── snapcraft.yaml
├── src/
│ ├── commands/
│ │ ├── auth/
│ │ │ ├── login.js
│ │ │ ├── logout.js
│ │ │ ├── register.js
│ │ │ └── whoami.js
│ │ ├── clone.js
│ │ ├── create.js
│ │ ├── doctor.js
│ │ ├── keys.js
│ │ ├── log.js
│ │ ├── publish.js
│ │ ├── pull.js
│ │ ├── status.js
│ │ ├── sync.js
│ │ └── unpublish.js
│ ├── extensions.js
│ ├── lib/
│ │ ├── archive.js
│ │ ├── discovery-exit.js
│ │ ├── download.js
│ │ ├── exit.js
│ │ ├── import-progress.js
│ │ ├── network.js
│ │ ├── selective-sync.js
│ │ ├── serve-http.js
│ │ └── stats.js
│ ├── parse-args.js
│ ├── registry.js
│ ├── ui/
│ │ ├── archive.js
│ │ ├── components/
│ │ │ ├── download.js
│ │ │ ├── import-progress.js
│ │ │ ├── network.js
│ │ │ ├── sources.js
│ │ │ └── warnings.js
│ │ ├── create.js
│ │ ├── elements/
│ │ │ ├── key.js
│ │ │ ├── pluralize.js
│ │ │ └── version.js
│ │ └── status.js
│ └── usage.js
└── test/
├── auth.js
├── clone.js
├── create.js
├── dat-node.js
├── doctor.js
├── fixtures/
│ ├── all_hour.csv
│ └── folder/
│ └── nested/
│ └── hello.txt
├── helpers/
│ ├── auth-server.js
│ ├── index.js
│ └── spawn.js
├── http.js
├── keys.js
├── pull.js
├── share.js
├── sync-owner.js
├── sync-remote.js
└── usage.js
SYMBOL INDEX (53 symbols across 36 files)
FILE: bin/cli.js
function alias (line 101) | function alias (argv) {
function syncShorthand (line 113) | function syncShorthand (opts) {
function showUsageOrRunExtension (line 149) | function showUsageOrRunExtension (opts, help, usageMessage) {
function exitInvalidNode (line 154) | function exitInvalidNode () {
FILE: scripts/auth-server.js
function close (line 9) | function close (cb) {
FILE: src/commands/auth/login.js
function login (line 19) | function login (opts) {
function exitErr (line 85) | function exitErr (err) {
FILE: src/commands/auth/logout.js
function logout (line 19) | function logout (opts) {
function exitErr (line 36) | function exitErr (err) {
FILE: src/commands/auth/register.js
function register (line 18) | function register (opts) {
function exitErr (line 89) | function exitErr (err) {
FILE: src/commands/auth/whoami.js
function whoami (line 18) | function whoami (opts) {
function exitErr (line 46) | function exitErr (err) {
FILE: src/commands/clone.js
function clone (line 32) | function clone (opts) {
FILE: src/commands/create.js
function create (line 28) | function create (opts) {
FILE: src/commands/keys.js
function keys (line 24) | function keys (opts) {
function run (line 42) | function run (dat, opts) {
function exit (line 106) | function exit (err) {
FILE: src/commands/publish.js
function publish (line 19) | function publish (opts) {
function exitErr (line 149) | function exitErr (err) {
FILE: src/commands/pull.js
function pull (line 45) | function pull (opts) {
FILE: src/commands/status.js
function status (line 12) | function status (opts) {
FILE: src/commands/sync.js
function sync (line 52) | function sync (opts) {
FILE: src/commands/unpublish.js
function unpublish (line 19) | function unpublish (opts) {
function exitErr (line 86) | function exitErr (err) {
FILE: src/extensions.js
function runExtension (line 6) | function runExtension (opts) {
FILE: src/lib/archive.js
function selectiveSync (line 35) | function selectiveSync (state, bus) {
FILE: src/lib/discovery-exit.js
function discoveryExit (line 5) | function discoveryExit (state, bus) {
FILE: src/lib/download.js
function trackDownload (line 6) | function trackDownload (state, bus) {
FILE: src/lib/exit.js
function onExit (line 4) | function onExit (state, bus) {
FILE: src/lib/import-progress.js
function trackImport (line 5) | function trackImport (state, bus) {
FILE: src/lib/network.js
function trackNetwork (line 10) | function trackNetwork (state, bus) {
FILE: src/lib/selective-sync.js
function parseFiles (line 17) | function parseFiles (input) {
FILE: src/lib/serve-http.js
function runHttp (line 4) | function runHttp (state, bus) {
FILE: src/lib/stats.js
function trackStats (line 5) | function trackStats (state, bus) {
FILE: src/ui/archive.js
function archiveUI (line 17) | function archiveUI (state) {
FILE: src/ui/components/download.js
function networkUI (line 6) | function networkUI (state) {
FILE: src/ui/components/import-progress.js
function importUI (line 8) | function importUI (state) {
FILE: src/ui/components/network.js
function networkUI (line 7) | function networkUI (state) {
FILE: src/ui/components/sources.js
function peersUI (line 7) | function peersUI (state) {
FILE: src/ui/create.js
function createUI (line 10) | function createUI (state) {
FILE: src/ui/status.js
function statusUI (line 8) | function statusUI (state) {
FILE: test/create.js
function doSync (line 67) | function doSync () {
FILE: test/helpers/auth-server.js
function createServer (line 13) | function createServer (port, cb) {
FILE: test/helpers/index.js
function shareFixtures (line 19) | function shareFixtures (opts, cb) {
function fileList (line 38) | function fileList (dir) {
function newTestFolder (line 46) | function newTestFolder () {
function matchDatLink (line 53) | function matchDatLink (str) {
function datJson (line 65) | function datJson (filepath) {
function isDir (line 73) | function isDir (dir) {
function fetchText (line 81) | function fetchText (url) {
FILE: test/keys.js
function exchangeKeys (line 67) | function exchangeKeys () {
FILE: test/pull.js
function doPull (line 37) | function doPull () {
Condensed preview — 72 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (172K chars).
[
{
"path": ".github/FUNDING.yml",
"chars": 21,
"preview": "open_collective: dat\n"
},
{
"path": ".github/issue_template.md",
"chars": 1192,
"preview": "\n<!--\nThanks for opening an issue! Please help us address your bug:\n\n- The issue tracker is only for bugs and feature re"
},
{
"path": ".gitignore",
"chars": 129,
"preview": "node_modules\n.DS_Store\ntmp\n.idea\ndata\nyarn.lock\ntest/fixtures/.dat\ntest/fixtures/dat.json\ntest/**.db\ntest/.datrc-test\ndi"
},
{
"path": ".travis.yml",
"chars": 693,
"preview": "language: node_js\n\nnode_js:\n - 'lts/*'\n - '12'\n - 'node'\nsudo: false\n\nscript:\n - npm test\n\nnotifications:\n irc:\n "
},
{
"path": "CODE_OF_CONDUCT.md",
"chars": 142,
"preview": "Our code of conduct is under review in [this repo](https://github.com/datproject/Code-of-Conduct) - please check it out "
},
{
"path": "CONTRIBUTING.md",
"chars": 6467,
"preview": "# Welcome to Dat!\n\nPlease take a second to read over this before opening an issue. Providing complete information upfron"
},
{
"path": "LICENSE",
"chars": 1479,
"preview": "Copyright (c) 2015 Max Ogden. All rights reserved.\nRedistribution and use in source and binary forms, with or without mo"
},
{
"path": "README.md",
"chars": 16912,
"preview": "More info on active projects and modules at [dat-ecosystem.org](https://dat-ecosystem.org/) <img src=\"https://i.imgur.co"
},
{
"path": "appveyor.yml",
"chars": 375,
"preview": "# Test against this version of Node.js\nenvironment:\n matrix:\n - nodejs_version: \"6\"\n - nodejs_version: \"8\"\n - "
},
{
"path": "bin/cli.js",
"chars": 4577,
"preview": "#!/usr/bin/env node\n\nvar subcommand = require('subcommand')\nvar debug = require('debug')('dat')\nvar usage = require('../"
},
{
"path": "changelog.md",
"chars": 9468,
"preview": "# Change Log\nAll notable changes to this project will be documented in this file.\nThis project adheres to [Semantic Vers"
},
{
"path": "download.sh",
"chars": 2009,
"preview": "#!/bin/bash\n\n# gets latest dat release zip for platform and extracts runnable binary into ~/.dat\n# usage: wget -qO- http"
},
{
"path": "index.js",
"chars": 37,
"preview": "module.exports = require('dat-node')\n"
},
{
"path": "package.json",
"chars": 2356,
"preview": "{\n \"name\": \"dat\",\n \"version\": \"14.0.3\",\n \"description\": \"Dat is the package manager for data. Easily share and versio"
},
{
"path": "package.sh",
"chars": 1454,
"preview": "#!/usr/bin/env sh\n# couldnt figure out undocumented 'output template' mode for pkg so wrote this\n# also need to include "
},
{
"path": "scripts/auth-server.js",
"chars": 307,
"preview": "var createServer = require('../tests/helpers/auth-server')\n\ncreateServer(process.env.PORT || 8888, function (err, server"
},
{
"path": "snap/snapcraft.yaml",
"chars": 558,
"preview": "name: dat\nversion: '13.11.4'\nsummary: Share & live sync files anywhere via command line\ndescription: |\n Use Dat command"
},
{
"path": "src/commands/auth/login.js",
"chars": 2083,
"preview": "module.exports = {\n name: 'login',\n command: login,\n help: [\n 'Login to a Dat registry server',\n 'Usage: dat lo"
},
{
"path": "src/commands/auth/logout.js",
"chars": 920,
"preview": "module.exports = {\n name: 'logout',\n command: logout,\n help: [\n 'Logout from current Dat registry server',\n 'Us"
},
{
"path": "src/commands/auth/register.js",
"chars": 2142,
"preview": "module.exports = {\n name: 'register',\n command: register,\n help: [\n 'Register with a public Dat registry',\n 'Us"
},
{
"path": "src/commands/auth/whoami.js",
"chars": 1133,
"preview": "module.exports = {\n name: 'whoami',\n command: whoami,\n help: [\n 'Get login information',\n 'Usage: dat login [<r"
},
{
"path": "src/commands/clone.js",
"chars": 4388,
"preview": "module.exports = {\n name: 'clone',\n command: clone,\n help: [\n 'Clone a remote Dat archive',\n '',\n 'Usage: da"
},
{
"path": "src/commands/create.js",
"chars": 3576,
"preview": "module.exports = {\n name: 'create',\n command: create,\n help: [\n 'Create an empty dat and dat.json',\n '',\n 'U"
},
{
"path": "src/commands/doctor.js",
"chars": 408,
"preview": "module.exports = {\n name: 'doctor',\n help: [\n 'Call the Doctor! Runs two tests:',\n ' 1. Check if you can connec"
},
{
"path": "src/commands/keys.js",
"chars": 2790,
"preview": "module.exports = {\n name: 'keys',\n command: keys,\n help: [\n 'View & manage dat keys',\n '',\n 'Usage:',\n ''"
},
{
"path": "src/commands/log.js",
"chars": 368,
"preview": "\nmodule.exports = {\n name: 'log',\n help: [\n 'View history and information about a dat',\n '',\n 'Usage: dat log"
},
{
"path": "src/commands/publish.js",
"chars": 5044,
"preview": "module.exports = {\n name: 'publish',\n command: publish,\n help: [\n 'Publish your dat to a Dat registry',\n 'Usage"
},
{
"path": "src/commands/pull.js",
"chars": 2615,
"preview": "module.exports = {\n name: 'pull',\n command: pull,\n help: [\n 'Pull updates from a cloned Dat archive',\n '',\n "
},
{
"path": "src/commands/status.js",
"chars": 1330,
"preview": "module.exports = {\n name: 'status',\n command: status,\n help: [\n 'Get information on about the Dat in a directory.'"
},
{
"path": "src/commands/sync.js",
"chars": 2371,
"preview": "module.exports = {\n name: 'sync',\n command: sync,\n help: [\n 'Sync a Dat archive with the network',\n 'Watch and "
},
{
"path": "src/commands/unpublish.js",
"chars": 2604,
"preview": "module.exports = {\n name: 'unpublish',\n command: unpublish,\n options: [\n {\n name: 'server',\n help: 'Unpu"
},
{
"path": "src/extensions.js",
"chars": 864,
"preview": "var debug = require('debug')('dat')\nvar os = require('os')\n\nmodule.exports = runExtension\n\nfunction runExtension (opts) "
},
{
"path": "src/lib/archive.js",
"chars": 2519,
"preview": "var debug = require('debug')('dat')\nvar path = require('path')\nvar EventEmitter = require('events').EventEmitter\nvar doI"
},
{
"path": "src/lib/discovery-exit.js",
"chars": 596,
"preview": "var output = require('neat-log/output')\n\nmodule.exports = discoveryExit\n\nfunction discoveryExit (state, bus) {\n bus.onc"
},
{
"path": "src/lib/download.js",
"chars": 1665,
"preview": "var debug = require('debug')('dat')\nvar xtend = Object.assign\n\nmodule.exports = trackDownload\n\nfunction trackDownload (s"
},
{
"path": "src/lib/exit.js",
"chars": 364,
"preview": "\nmodule.exports = onExit\n\nfunction onExit (state, bus) {\n bus.on('exit:error', onError)\n bus.on('exit:warn', function "
},
{
"path": "src/lib/import-progress.js",
"chars": 1680,
"preview": "var xtend = Object.assign\n\nmodule.exports = trackImport\n\nfunction trackImport (state, bus) {\n if (state.dat) return tra"
},
{
"path": "src/lib/network.js",
"chars": 2902,
"preview": "var bytes = require('bytes').parse\nvar speed = require('speedometer')\nvar throttle = require('throttle')\nvar pump = requ"
},
{
"path": "src/lib/selective-sync.js",
"chars": 819,
"preview": "var fs = require('fs')\nvar path = require('path')\n\nmodule.exports = function (state, opts) {\n // selective sync stuff\n "
},
{
"path": "src/lib/serve-http.js",
"chars": 402,
"preview": "\nmodule.exports = runHttp\n\nfunction runHttp (state, bus) {\n if (state.dat) return serve()\n bus.once('dat', serve)\n\n f"
},
{
"path": "src/lib/stats.js",
"chars": 392,
"preview": "var xtend = Object.assign\n\nmodule.exports = trackStats\n\nfunction trackStats (state, bus) {\n if (state.dat) return track"
},
{
"path": "src/parse-args.js",
"chars": 1154,
"preview": "var fs = require('fs')\nvar path = require('path')\nvar encoding = require('dat-encoding')\n\nmodule.exports = function (opt"
},
{
"path": "src/registry.js",
"chars": 494,
"preview": "var xtend = Object.assign\nvar RegistryClient = require('dat-registry')\n\nmodule.exports = function (opts) {\n var townshi"
},
{
"path": "src/ui/archive.js",
"chars": 2169,
"preview": "var path = require('path')\nvar output = require('neat-log/output')\nvar pretty = require('prettier-bytes')\nvar chalk = re"
},
{
"path": "src/ui/components/download.js",
"chars": 1239,
"preview": "var output = require('neat-log/output')\nvar bar = require('progress-string')\n\nmodule.exports = networkUI\n\nfunction netwo"
},
{
"path": "src/ui/components/import-progress.js",
"chars": 2093,
"preview": "var output = require('neat-log/output')\nvar pretty = require('prettier-bytes')\nvar bar = require('progress-string')\nvar "
},
{
"path": "src/ui/components/network.js",
"chars": 717,
"preview": "var output = require('neat-log/output')\nvar pretty = require('prettier-bytes')\nvar pluralize = require('../elements/plur"
},
{
"path": "src/ui/components/sources.js",
"chars": 989,
"preview": "var output = require('neat-log/output')\nvar pretty = require('prettier-bytes')\nvar makeBar = require('progress-string')\n"
},
{
"path": "src/ui/components/warnings.js",
"chars": 213,
"preview": "var chalk = require('chalk')\n\nmodule.exports = function (state) {\n var warning = ''\n state.warnings.forEach(function ("
},
{
"path": "src/ui/create.js",
"chars": 1273,
"preview": "var output = require('neat-log/output')\nvar pretty = require('prettier-bytes')\nvar chalk = require('chalk')\nvar importUI"
},
{
"path": "src/ui/elements/key.js",
"chars": 164,
"preview": "var stringKey = require('dat-encoding').toStr\nvar chalk = require('chalk')\n\nmodule.exports = function (key) {\n return `"
},
{
"path": "src/ui/elements/pluralize.js",
"chars": 92,
"preview": "module.exports = function pluralize (str, val) {\n return `${str}${val === 1 ? '' : 's'}`\n}\n"
},
{
"path": "src/ui/elements/version.js",
"chars": 115,
"preview": "var chalk = require('chalk')\n\nmodule.exports = function (version) {\n return `${chalk.green(`dat v${version}`)}`\n}\n"
},
{
"path": "src/ui/status.js",
"chars": 486,
"preview": "var output = require('neat-log/output')\nvar stringKey = require('dat-encoding').toStr\nvar pretty = require('prettier-byt"
},
{
"path": "src/usage.js",
"chars": 1136,
"preview": "module.exports = function (opts, help, usage) {\n if (opts.version) {\n var pkg = require('../package.json')\n conso"
},
{
"path": "test/auth.js",
"chars": 7386,
"preview": "var test = require('tape')\nvar path = require('path')\nvar fs = require('fs')\nvar rimraf = require('rimraf')\nvar mkdirp ="
},
{
"path": "test/clone.js",
"chars": 16418,
"preview": "var fs = require('fs')\nvar path = require('path')\nvar test = require('tape')\nvar tempDir = require('temporary-directory'"
},
{
"path": "test/create.js",
"chars": 3469,
"preview": "var fs = require('fs')\nvar path = require('path')\nvar test = require('tape')\nvar tempDir = require('temporary-directory'"
},
{
"path": "test/dat-node.js",
"chars": 284,
"preview": "var test = require('tape')\nvar ram = require('random-access-memory')\nvar Dat = require('..')\n\ntest('dat-node: require da"
},
{
"path": "test/doctor.js",
"chars": 1570,
"preview": "// var path = require('path')\n// var test = require('tape')\n// var spawn = require('./helpers/spawn.js')\n// var help = r"
},
{
"path": "test/fixtures/all_hour.csv",
"chars": 1441,
"preview": "time,latitude,longitude,depth,mag,magType,nst,gap,dmin,rms,net,id,updated,place,type\n2014-04-30T03:34:57.000Z,60.0366,-1"
},
{
"path": "test/fixtures/folder/nested/hello.txt",
"chars": 28,
"preview": "code for science and society"
},
{
"path": "test/helpers/auth-server.js",
"chars": 1412,
"preview": "var path = require('path')\nvar rimraf = require('rimraf')\nvar Server, initDb\ntry {\n Server = require('dat-registry-api/"
},
{
"path": "test/helpers/index.js",
"chars": 2265,
"preview": "var fs = require('fs')\nvar os = require('os')\nvar path = require('path')\nvar mkdirp = require('mkdirp')\nvar rimraf = req"
},
{
"path": "test/helpers/spawn.js",
"chars": 352,
"preview": "var spawn = require('tape-spawn')\nvar fs = require('fs')\n\n// happens once at require time\n// https://github.com/AndreasM"
},
{
"path": "test/http.js",
"chars": 1985,
"preview": "var path = require('path')\nvar test = require('tape')\nvar rimraf = require('rimraf')\nvar spawn = require('./helpers/spaw"
},
{
"path": "test/keys.js",
"chars": 3235,
"preview": "var fs = require('fs')\nvar path = require('path')\nvar test = require('tape')\nvar rimraf = require('rimraf')\nvar tempDir "
},
{
"path": "test/pull.js",
"chars": 4221,
"preview": "var path = require('path')\nvar test = require('tape')\nvar tempDir = require('temporary-directory')\nvar spawn = require('"
},
{
"path": "test/share.js",
"chars": 1862,
"preview": "// var fs = require('fs')\n// var path = require('path')\n// var test = require('tape')\n// var rimraf = require('rimraf')\n"
},
{
"path": "test/sync-owner.js",
"chars": 8395,
"preview": "// var fs = require('fs')\n// var net = require('net')\n// var path = require('path')\n// var test = require('tape')\n// var"
},
{
"path": "test/sync-remote.js",
"chars": 2883,
"preview": "// var path = require('path')\n// var test = require('tape')\n// var rimraf = require('rimraf')\n// var spawn = require('./"
},
{
"path": "test/usage.js",
"chars": 1114,
"preview": "var path = require('path')\nvar test = require('tape')\nvar spawn = require('./helpers/spawn.js')\n\nvar dat = path.resolve("
}
]
About this extraction
This page contains the full source code of the dat-ecosystem/dat GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 72 files (159.0 KB), approximately 43.9k tokens, and a symbol index with 53 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.