* [Horizon Repo Examples Directory](https://github.com/rethinkdb/horizon/tree/next/examples)
* [CycleJS Chat App](https://github.com/rethinkdb/horizon/tree/next/examples/cyclejs-chat-app)
* [RiotJS Chat App](https://github.com/rethinkdb/horizon/tree/next/examples/riotjs-chat-app)
* [React Chat App](https://github.com/rethinkdb/horizon/tree/next/examples/react-chat-app)
* [React TodoMVC App](https://github.com/rethinkdb/horizon/tree/next/examples/react-todo-app)
* [Vue Chat App](https://github.com/rethinkdb/horizon/tree/next/examples/vue-chat-app)
* [Vue TodoMVC App](https://github.com/rethinkdb/horizon/tree/next/examples/vue-todo-app)
## Extending Horizon Server
We also have a few examples of how you can extend Horizon Server. We imagine that once your application
grows beyond the needs of simply providing the Horizon Client API, you'll want to expand and build upon
Horizon Server. Here are a few examples of how to extend Horizon Server with some popular Node web frameworks.
* [Extending with Koa Server](https://github.com/rethinkdb/horizon/tree/next/examples/koa-server)
* [Extending with Hapi Server](https://github.com/rethinkdb/horizon/tree/next/examples/hapi-server)
* [Extending with Express Server](https://github.com/rethinkdb/horizon/tree/next/examples/express-server)
[above]: https://github.com/rethinkdb/horizon/tree/next/client#above-limit-integer--key-value-closed-string-
[below]: https://github.com/rethinkdb/horizon/tree/next/client#below-limit-integer--key-value-closed-string-
[Collection]: https://github.com/rethinkdb/horizon/tree/next/client#collection
[fetch]: https://github.com/rethinkdb/horizon/tree/next/client#fetch
[find]: https://github.com/rethinkdb/horizon/tree/next/client#find---id-any-
[findAll]: https://github.com/rethinkdb/horizon/tree/next/client#findall--id-any----id-any--
[Horizon]: https://github.com/rethinkdb/horizon/tree/next/client#horizon
[limit]: https://github.com/rethinkdb/horizon/tree/next/client#limit-num-integer-
[order]: https://github.com/rethinkdb/horizon/tree/next/client#order---directionascending-
[remove]: https://github.com/rethinkdb/horizon/tree/next/client#remove-id-any--id-any-
[removeAll]: https://github.com/rethinkdb/horizon/tree/next/client#removeall--id-any--id-any-----id-any---id-any---
[replace]: https://github.com/rethinkdb/horizon/tree/next/client#replace--
[store]: https://github.com/rethinkdb/horizon/tree/next/client#store-------
[store]: https://github.com/rethinkdb/horizon/tree/next/client#store-------
[upsert]: https://github.com/rethinkdb/horizon/tree/next/client#upsert------
[watch]: https://github.com/rethinkdb/horizon/tree/next/client#watch--rawchanges-false--
================================================
FILE: ISSUE_TEMPLATE.md
================================================
If you're reporting a bug please include the server version and client version.
================================================
FILE: LICENSE
================================================
The MIT License (MIT)
Copyright (c) 2016 RethinkDB, Inc.
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
================================================
FILE: README.md
================================================
# Horizon
[Official Repository](https://github.com/rethinkdb/horizon)
## What is Horizon?
Horizon is an open-source developer platform for building sophisticated realtime
apps. It provides a complete backend that makes it dramatically simpler to
build, deploy, manage, and scale engaging JavaScript web and mobile apps.
Horizon is extensible, integrates with the Node.js stack, and allows building
modern, arbitrarily complex applications.
Horizon is built on top of [RethinkDB](https://www.rethinkdb.com) and consists of
four components:
- [__Horizon server__](/server) -- a middleware server that connects to/is built on
top of RethinkDB, and exposes a simple API/protocol to front-end
applications.
- [__Horizon client library__](/client) -- a JavaScript client library that wraps
Horizon server's protocol in a convenient API for front-end
developers.
- [__Horizon CLI - `hz`__](/cli) -- a command-line tool aiding in scaffolding, development, and deployment
- [__GraphQL support__](https://github.com/rethinkdb/horizon/issues/125) -- the server will have a GraphQL adapter so anyone can get started building React/Relay apps without writing any backend code at the beginning. This will not ship in v1, but we'll follow up with a GraphQL adapter quickly after launch.
Horizon currently has all the following services available to developers:
- ✅ __Subscribe__ -- a streaming API for building realtime apps directly from the
browser without writing any backend code.
- ✅ __Auth__ -- an authentication API that connects to common auth providers
(e.g. Facebook, Google, GitHub).
- ✅ __Identity__ -- an API for listing and manipulating user accounts.
- ✅ __Permissions__ -- a security model that allows the developer to protect
data from unauthorized access.
Upcoming versions of Horizon will likely expose the following
additional services:
- __Session management__ -- manage browser session and session
information.
- __Geolocation__ -- an API that makes it very easy to build
location-aware apps.
- __Presence__ -- an API for detecting presence information for a given
user and sharing it with others.
- __Plugins__ -- a system for extending Horizon with user-defined services
in a consistent, discoverable way.
- __Backend__ -- an API/protocol to integrate custom backend code with
Horizon server/client-libraries.
## Why Horizon?
While technologies like [RethinkDB](http://www.rethinkdb.com) and
[WebSocket](https://en.wikipedia.org/wiki/WebSocket) make it possible to build
engaging realtime apps, empirically there is still too much friction for most
developers. Building realtime apps now requires understanding and manually
orchestrating multiple systems across the software stack, understanding
distributed stream processing, and learning how to deploy and scale realtime systems. The
learning curve is quite steep, and most of the initial work involves boilerplate
code that is far removed from the primary task of building a realtime app.
Horizon sets out to solve this problem. Developers can start building
apps using their favorite front-end framework using Horizon's APIs
without having to write any backend code.
Since Horizon stores data in RethinkDB, once the app gets sufficiently
complex to need custom business logic on the backend, developers can
incrementally add backend code at any time in the development cycle of
their app.
## Get Involved
We'd love for you to help us build Horizon. If you'd like to be a contributor,
check out our [Contributing guide](/CONTRIBUTING.md).
Also, to stay up-to-date on all Horizon related news and the community you should
definitely [join us on Slack](http://slack.rethinkdb.com) or [follow us on Twitter](https://twitter.com/horizonjs).
## FAQ
Check out our FAQ at [horizon.io/faq](https://horizon.io/faq/)
### How will Horizon be licensed?
The Horizon server, client and cli are available under the MIT license
================================================
FILE: circle.yml
================================================
## Customize the test machine
machine:
#timezone:
# America/Los_Angeles # Set the timezone
# Set version of node to use
#node:
# version:
# 5.7.0
post:
- source /etc/lsb-release && echo "deb http://download.rethinkdb.com/apt $DISTRIB_CODENAME main" | sudo tee /etc/apt/sources.list.d/rethinkdb.list
- wget -qO- https://download.rethinkdb.com/apt/pubkey.gpg | sudo apt-key add -
- sudo apt-get update -o Dir::Etc::sourcelist="/etc/apt/sources.list.d/rethinkdb.list" -o Dir::Etc::sourceparts="-" -o APT::Get::List-Cleanup="0"
- sudo apt-get install rethinkdb
## Set artifacts
# general:
# artifacts:
# - "client/npm-debug.log"
# - "server/npm-debug.log"
# - "cli/npm-debug.log"
## Customize dependencies
dependencies:
# Cache directories for speed
cache_directories:
- client/node_modules
- server/node_modules
- cli/node_modules
override:
# Stop default services
#- sudo service redis-server stop
#- sudo service postgresql stop
#- sudo service mysql stop
# Prepare for client tests
#- npm prune --production:
# pwd: client
# Prepare for server tests
#- npm prune --production:
# pwd: server
#- npm prune --production:
# pwd: cli
- ./setupDev.sh:
pwd: test
## Customize test commands
test:
pre:
- ./test/serve.js:
background: true
# - mkdir -p $CIRCLE_TEST_REPORTS/xunit
# - touch $CIRCLE_TEST_REPORTS/xunit/cli-tests.xml
# - touch $CIRCLE_TEST_REPORTS/xunit/client-tests.xml
# - touch $CIRCLE_TEST_REPORTS/xunit/server-tests.xml
override:
# Run client tests
- ./node_modules/.bin/mocha --timeout 100000 dist/test.js:
pwd: client
parallel: false
# Run server tests
- ./node_modules/.bin/mocha --timeout 100000 test/test.js test/schema.js:
pwd: server
parallel: false
# Run cli tests
- ./node_modules/.bin/mocha --timeout 100000 test:
pwd: cli
parallel: false
================================================
FILE: cli/.eslintrc.js
================================================
const OFF = 0;
const WARN = 1;
const ERROR = 2;
module.exports = {
extends: "../.eslintrc.js",
rules: {
"max-len": [ ERROR, 100 ],
"no-invalid-this": [ OFF ]
},
env: {
"es6": true,
"node": true,
"mocha": true,
},
};
================================================
FILE: cli/.gitignore
================================================
# Coverage directory used by tools like istanbul
coverage
================================================
FILE: cli/README.md
================================================
# **Horizon** is a realtime, open-source backend for JavaScript apps.
Rapidly build and deploy web or mobile apps using a simple JavaScript API. Scale your apps to millions of users without any backend code.
Horizon consists of three components:
* Horizon server: a middleware server that connects to/is built on top of RethinkDB, and exposes a simple API/protocol to front-end applications.
* Horizon client: a JavaScript client library that wraps Horizon server’s protocol in a convenient API for front-end developers.
* Horizon CLI: a command line tool, hz, aiding in scaffolding, development, and deployment.
Built by the [RethinkDB](https://rethinkdb.com) team and an open-source community, Horizon lets you build sophisticated apps with lightning speed.
## Installing Horizon
https://horizon.io/install/
## Getting Started
https://horizon.io/docs/getting-started/
================================================
FILE: cli/package.json
================================================
{
"name": "horizon",
"version": "2.0.0",
"description": "An open-source developer platform for building realtime, scalable web apps.",
"main": "src/main.js",
"repository": {
"type": "git",
"url": "git+https://github.com/rethinkdb/horizon.git"
},
"scripts": {
"lint": "eslint src test",
"test": "mocha test test/unit --timeout 10000",
"coverage": "istanbul cover _mocha test"
},
"author": "RethinkDB",
"license": "MIT",
"bin": {
"hz": "src/main.js",
"horizon": "src/main.js"
},
"bugs": {
"url": "https://github.com/rethinkdb/horizon/issues"
},
"homepage": "https://github.com/rethinkdb/horizon#readme",
"dependencies": {
"@horizon/server": "2.0.0",
"argparse": "^1.0.3",
"bluebird": "^3.4.1",
"chalk": "^1.1.3",
"hasbin": "^1.2.1",
"joi": "^8.0.5",
"jsonwebtoken": "^5.5.4",
"mime-types": "^2.0.4",
"open": "7.0.4",
"rethinkdb": "^2.1.1",
"toml": "^2.3.0"
},
"devDependencies": {
"chai": "^3.5.0",
"eslint": "^7.3.1",
"istanbul": "^0.4.3",
"mocha": "2.4.5",
"mock-fs": "^3.10.0",
"sinon": "1.17.3",
"strip-ansi": "^3.0.1",
"toml": "^2.3.0"
},
"engines": {
"node": ">=4.0.0",
"npm": ">=3.0.0"
},
"preferGlobal": true
}
================================================
FILE: cli/src/create-cert.js
================================================
'use strict';
const hasbin = require('hasbin');
const spawn = require('child_process').spawn;
const run = (args) => {
if (args.length) {
throw new Error('create-cert takes no arguments');
}
// TODO: user configuration?
const settings = {
binaryName: 'openssl',
keyOutName: 'horizon-key.pem',
certOutName: 'horizon-cert.pem',
algo: 'rsa',
bits: '2048',
days: '365',
};
// generate the arguments to the command
const binArgs = [ 'req', '-x509', '-nodes', '-batch',
'-newkey', `${settings.algo}:${settings.bits}`,
'-keyout', settings.keyOutName,
'-out', settings.certOutName,
'-days', settings.days,
];
return new Promise((resolve, reject) => {
hasbin(settings.binaryName, (hasOpenSSL) => {
// show the invocation that's about to be run
console.log(`> ${settings.binaryName} ${binArgs.join(' ')}`);
// if we don't have openssl, bail
if (!hasOpenSSL) {
reject(new Error(`Missing ${settings.binaryName}. Make sure it is on the path.`));
}
// otherwise start openssl
const sslProc = spawn(settings.binaryName, binArgs);
// pipe output appropriately
sslProc.stdout.pipe(process.stdout, { end: false });
sslProc.stderr.pipe(process.stderr, { end: false });
// say nice things to the user when it's done
sslProc.on('error', reject);
sslProc.on('close', (code) => {
if (code) {
reject(new Error(`OpenSSL failed with code ${code}.`));
} else {
console.log('Everything seems to be fine. ' +
'Remember to add your shiny new certificates to your Horizon config!');
resolve();
}
});
});
});
};
module.exports = {
run,
description: 'Generate a certificate',
};
================================================
FILE: cli/src/init.js
================================================
/* global require, module */
'use strict';
const fs = require('fs');
const crypto = require('crypto');
const process = require('process');
const argparse = require('argparse');
const checkProjectName = require('./utils/check-project-name');
const rethrow = require('./utils/rethrow');
const makeIndexHTML = (projectName) => `\
`;
const makeDefaultConfig = (projectName) => `\
# This is a TOML file
###############################################################################
# IP options
# 'bind' controls which local interfaces will be listened on
# 'port' controls which port will be listened on
#------------------------------------------------------------------------------
# bind = [ "localhost" ]
# port = 8181
###############################################################################
# HTTPS Options
# 'secure' will disable HTTPS and use HTTP instead when set to 'false'
# 'key_file' and 'cert_file' are required for serving HTTPS
#------------------------------------------------------------------------------
# secure = true
# key_file = "horizon-key.pem"
# cert_file = "horizon-cert.pem"
###############################################################################
# App Options
# 'project_name' sets the name of the RethinkDB database used to store the
# application state
# 'serve_static' will serve files from the given directory over HTTP/HTTPS
#------------------------------------------------------------------------------
project_name = "${projectName}"
# serve_static = "dist"
###############################################################################
# Data Options
# WARNING: these should probably not be enabled on a publically accessible
# service. Tables and indexes are not lightweight objects, and allowing them
# to be created like this could open the service up to denial-of-service
# attacks.
# 'auto_create_collection' creates a collection when one is needed but does not exist
# 'auto_create_index' creates an index when one is needed but does not exist
#------------------------------------------------------------------------------
# auto_create_collection = false
# auto_create_index = false
###############################################################################
# RethinkDB Options
# 'connect' and 'start_rethinkdb' are mutually exclusive
# 'connect' will connect to an existing RethinkDB instance
# 'start_rethinkdb' will run an internal RethinkDB instance
# 'rdb_timeout' is the number of seconds to wait when connecting to RethinkDB
#------------------------------------------------------------------------------
# connect = "localhost:28015"
# start_rethinkdb = false
# rdb_timeout = 30
###############################################################################
# Debug Options
# 'debug' enables debug log statements
#------------------------------------------------------------------------------
# debug = false
###############################################################################
# Authentication Options
# Each auth subsection will add an endpoint for authenticating through the
# specified provider.
# 'token_secret' is the key used to sign jwts
# 'allow_anonymous' issues new accounts to users without an auth provider
# 'allow_unauthenticated' allows connections that are not tied to a user id
# 'auth_redirect' specifies where users will be redirected to after login
# 'access_control_allow_origin' sets a host that can access auth settings
# (typically your frontend host)
#------------------------------------------------------------------------------
# allow_anonymous = false
# allow_unauthenticated = false
# auth_redirect = "/"
# access_control_allow_origin = ""
#
`;
const makeDefaultSchema = () => `\
[groups.admin]
[groups.admin.rules.carte_blanche]
template = "any()"
`;
const makeDefaultSecrets = () => `\
token_secret = "${crypto.randomBytes(64).toString('base64')}"
###############################################################################
# RethinkDB Options
# 'rdb_user' is the user account to log in with when connecting to RethinkDB
# 'rdb_password' is the password for the user account specified by 'rdb_user'
#------------------------------------------------------------------------------
# rdb_user = 'admin'
# rdb_password = ''
# [auth.auth0]
# host = "0000.00.auth0.com"
# id = "0000000000000000000000000"
# secret = "00000000000000000000000000000000000000000000000000"
#
# [auth.facebook]
# id = "000000000000000"
# secret = "00000000000000000000000000000000"
#
# [auth.google]
# id = "00000000000-00000000000000000000000000000000.apps.googleusercontent.com"
# secret = "000000000000000000000000"
#
# [auth.twitter]
# id = "0000000000000000000000000"
# secret = "00000000000000000000000000000000000000000000000000"
#
# [auth.github]
# id = "00000000000000000000"
# secret = "0000000000000000000000000000000000000000"
#
# [auth.twitch]
# id = "0000000000000000000000000000000"
# secret = "0000000000000000000000000000000"
#
# [auth.slack]
# id = "0000000000000000000000000000000"
# secret = "0000000000000000000000000000000"
`;
const gitignore = () => `\
rethinkdb_data
**/*.log
.hz/secrets.toml
node_modules
`;
const parseArguments = (args) => {
const parser = new argparse.ArgumentParser({ prog: 'hz init' });
parser.addArgument([ 'projectName' ],
{ action: 'store',
help: 'Name of directory to create. Defaults to current directory',
nargs: '?',
}
);
return parser.parseArgs(args);
};
const fileExists = (pathName) => {
try {
fs.statSync(pathName);
return true;
} catch (e) {
return false;
}
};
const maybeMakeDir = (createDir, dirName) => {
if (createDir) {
try {
fs.mkdirSync(dirName);
console.info(`Created new project directory ${dirName}`);
} catch (e) {
throw rethrow(e,
`Couldn't make directory ${dirName}: ${e.message}`);
}
} else {
console.info(`Initializing in existing directory ${dirName}`);
}
};
const maybeChdir = (chdirTo) => {
if (chdirTo) {
try {
process.chdir(chdirTo);
} catch (e) {
if (e.code === 'ENOTDIR') {
throw rethrow(e, `${chdirTo} is not a directory`);
} else {
throw rethrow(e, `Couldn't chdir to ${chdirTo}: ${e.message}`);
}
}
}
};
const populateDir = (projectName, dirWasPopulated, chdirTo, dirName) => {
const niceDir = chdirTo ? `${dirName}/` : '';
if (!dirWasPopulated && !fileExists('src')) {
fs.mkdirSync('src');
console.info(`Created ${niceDir}src directory`);
}
if (!dirWasPopulated && !fileExists('dist')) {
fs.mkdirSync('dist');
console.info(`Created ${niceDir}dist directory`);
fs.appendFileSync('./dist/index.html', makeIndexHTML(projectName));
console.info(`Created ${niceDir}dist/index.html example`);
}
if (!fileExists('.hz')) {
fs.mkdirSync('.hz');
console.info(`Created ${niceDir}.hz directory`);
}
// Default permissions
const permissionGeneral = {
encoding: 'utf8',
mode: 0o666,
};
const permissionSecret = {
encoding: 'utf8',
mode: 0o600, // Secrets are put in this config, so set it user, read/write only
};
// Create .gitignore if it doesn't exist
if (!fileExists('.gitignore')) {
fs.appendFileSync(
'.gitignore',
gitignore(),
permissionGeneral
);
console.info(`Created ${niceDir}.gitignore`);
} else {
console.info('.gitignore already exists, not touching it.');
}
// Create .hz/config.toml if it doesn't exist
if (!fileExists('.hz/config.toml')) {
fs.appendFileSync(
'.hz/config.toml',
makeDefaultConfig(projectName),
permissionGeneral
);
console.info(`Created ${niceDir}.hz/config.toml`);
} else {
console.info('.hz/config.toml already exists, not touching it.');
}
// Create .hz/schema.toml if it doesn't exist
if (!fileExists('.hz/schema.toml')) {
fs.appendFileSync(
'.hz/schema.toml',
makeDefaultSchema(),
permissionGeneral
);
console.info(`Created ${niceDir}.hz/schema.toml`);
} else {
console.info('.hz/schema.toml already exists, not touching it.');
}
// Create .hz/secrets.toml if it doesn't exist
if (!fileExists('.hz/secrets.toml')) {
fs.appendFileSync(
'.hz/secrets.toml',
makeDefaultSecrets(),
permissionSecret
);
console.info(`Created ${niceDir}.hz/secrets.toml`);
} else {
console.info('.hz/secrets.toml already exists, not touching it.');
}
};
const run = (args) =>
Promise.resolve(args)
.then(parseArguments)
.then((parsed) => {
const check = checkProjectName(
parsed.projectName,
process.cwd(),
fs.readdirSync('.')
);
const projectName = check.projectName;
const dirName = check.dirName;
const chdirTo = check.chdirTo;
const createDir = check.createDir;
maybeMakeDir(createDir, dirName);
maybeChdir(chdirTo);
// Before we create things, check if the directory is empty
const dirWasPopulated = fs.readdirSync(process.cwd()).length !== 0;
populateDir(projectName, dirWasPopulated, chdirTo, dirName);
});
module.exports = {
run,
description: 'Initialize a horizon app directory',
};
================================================
FILE: cli/src/main.js
================================================
#!/usr/bin/env node
'use strict';
// To support `pidof horizon`, by default it shows in `pidof node`
process.title = 'horizon';
const chalk = require('chalk');
const path = require('path');
const initCommand = require('./init');
const serveCommand = require('./serve');
const versionCommand = require('./version');
const createCertCommand = require('./create-cert');
const schemaCommand = require('./schema');
const makeTokenCommand = require('./make-token');
const migrateCommand = require('./migrate');
const NiceError = require('./utils/nice_error');
// Mapping from command line strings to modules. To add a new command,
// add an entry in this object, and create a module with the following
// exported:
// - run: main function for the command
// - description: a string to display in the hz help text
const commands = {
init: initCommand,
serve: serveCommand,
version: versionCommand,
'create-cert': createCertCommand,
'make-token': makeTokenCommand,
schema: schemaCommand,
migrate: migrateCommand,
};
const programName = path.basename(process.argv[1]);
const help = () => {
console.log(`Usage: ${programName} subcommand [args...]`);
console.log('Available subcommands:');
Object.keys(commands).forEach((cmdName) =>
console.log(` ${cmdName} - ${commands[cmdName].description}`)
);
};
const allArgs = process.argv.slice(2);
if (allArgs.length === 0) {
help();
process.exit(1);
}
const cmdName = allArgs[0];
const cmdArgs = allArgs.slice(1);
if (cmdName === '-h' || cmdName === '--help' || cmdName === 'help') {
help();
process.exit(0);
}
const command = commands[cmdName];
if (!command) {
console.error(chalk.red.bold(
`No such subcommand ${cmdName}, run with -h for help`));
process.exit(1);
}
const done = (err) => {
if (err) {
const errMsg = (err instanceof NiceError) ?
err.niceString({ contextSize: 2 }) : err.message;
console.error(chalk.red.bold(errMsg));
process.exit(1);
} else {
process.exit(0);
}
};
try {
command.run(cmdArgs).then(() => done()).catch(done);
} catch (err) {
done(err);
}
================================================
FILE: cli/src/make-token.js
================================================
'use strict';
const interrupt = require('./utils/interrupt');
const config = require('./utils/config');
const horizon_server = require('@horizon/server');
const path = require('path');
const jwt = require('jsonwebtoken');
const r = horizon_server.r;
const logger = horizon_server.logger;
const argparse = require('argparse');
const parseArguments = (args) => {
const parser = new argparse.ArgumentParser({ prog: 'hz make-token' });
parser.addArgument(
[ '--token-secret' ],
{ type: 'string', metavar: 'SECRET',
help: 'Secret key for signing the token.' });
parser.addArgument(
[ 'user' ],
{ type: 'string', metavar: 'USER_ID',
help: 'The ID of the user to issue a token for.' });
return parser.parseArgs(args);
};
const processConfig = (parsed) => {
let options;
options = config.default_options();
options = config.merge_options(
options, config.read_from_config_file(parsed.project_path));
options = config.merge_options(
options, config.read_from_secrets_file(parsed.project_path));
options = config.merge_options(options, config.read_from_env());
options = config.merge_options(options, config.read_from_flags(parsed));
if (options.project_name === null) {
options.project_name = path.basename(path.resolve(options.project_path));
}
return Object.assign(options, { user: parsed.user });
};
const run = (args) => Promise.resolve().then(() => {
const options = processConfig(parseArguments(args));
if (options.token_secret === null) {
throw new Error('No token secret specified, unable to sign the token.');
}
const token = jwt.sign(
{ id: options.user, provider: null },
new Buffer(options.token_secret, 'base64'),
{ expiresIn: '1d', algorithm: 'HS512' }
);
console.log(`${token}`);
});
module.exports = {
run,
description: 'Generate a token to log in as a user',
};
================================================
FILE: cli/src/migrate.js
================================================
'use strict';
const chalk = require('chalk');
const r = require('rethinkdb');
const Promise = require('bluebird');
const argparse = require('argparse');
const runSaveCommand = require('./schema').runSaveCommand;
const fs = require('fs');
const accessAsync = Promise.promisify(fs.access);
const config = require('./utils/config');
const procPromise = require('./utils/proc-promise');
const interrupt = require('./utils/interrupt');
const change_to_project_dir = require('./utils/change_to_project_dir');
const parse_yes_no_option = require('./utils/parse_yes_no_option');
const start_rdb_server = require('./utils/start_rdb_server');
const NiceError = require('./utils/nice_error.js');
const VERSION_2_0 = [ 2, 0, 0 ];
function run(cmdArgs) {
const options = processConfig(cmdArgs);
interrupt.on_interrupt(() => teardown());
return Promise.resolve().bind({ options })
.then(setup)
.then(validateMigration)
.then(makeBackup)
.then(renameUserTables)
.then(moveInternalTables)
.then(renameIndices)
.then(rewriteHzCollectionDocs)
.then(exportNewSchema)
.finally(teardown);
}
function green() {
const args = Array.from(arguments);
args[0] = chalk.green(args[0]);
console.log.apply(console, args);
}
function white() {
const args = Array.from(arguments);
args[0] = chalk.white(args[0]);
console.log.apply(console, args);
}
function processConfig(cmdArgs) {
// do boilerplate to get config args :/
const parser = new argparse.ArgumentParser({ prog: 'hz migrate' });
parser.addArgument([ 'project_path' ], {
default: '.',
nargs: '?',
help: 'Change to this directory before migrating',
});
parser.addArgument([ '--project-name', '-n' ], {
help: 'Name of the Horizon project server',
});
parser.addArgument([ '--connect', '-c' ], {
metavar: 'host:port',
default: undefined,
help: 'Host and port of the RethinkDB server to connect to.',
});
parser.addArgument([ '--rdb-user' ], {
default: 'admin',
metavar: 'USER',
help: 'RethinkDB User',
});
parser.addArgument([ '--rdb-password' ], {
default: undefined,
metavar: 'PASSWORD',
help: 'RethinkDB Password',
});
parser.addArgument([ '--start-rethinkdb' ], {
metavar: 'yes|no',
default: 'yes',
constant: 'yes',
nargs: '?',
help: 'Start up a RethinkDB server in the current directory',
});
parser.addArgument([ '--skip-backup' ], {
metavar: 'yes|no',
default: 'no',
constant: 'yes',
nargs: '?',
help: 'Whether to perform a backup of rethinkdb_data' +
' before migrating',
});
parser.addArgument([ '--nonportable-backup' ], {
metavar: 'yes|no',
default: 'no',
constant: 'yes',
nargs: '?',
help: 'Allows creating a backup that is not portable, ' +
"but doesn't require the RethinkDB Python driver to be " +
'installed.',
});
const parsed = parser.parseArgs(cmdArgs);
const confOptions = config.read_from_config_file(parsed.project_path);
const envOptions = config.read_from_env();
config.merge_options(confOptions, envOptions);
// Pull out the relevant settings from the config file
const options = {
project_path: parsed.project_path || '.',
project_name: parsed.project_name || confOptions.project_name,
rdb_host: parsed.rdb_host || confOptions.rdb_host || 'localhost',
rdb_port: parsed.rdb_port || confOptions.rdb_port || 28015,
rdb_user: parsed.rdb_user || confOptions.rdb_user || 'admin',
rdb_password: parsed.rdb_password || confOptions.rdb_password || '',
start_rethinkdb: parse_yes_no_option(parsed.start_rethinkdb),
skip_backup: parse_yes_no_option(parsed.skip_backup),
nonportable_backup: parse_yes_no_option(parsed.nonportable_backup),
};
// sets rdb_host and rdb_port from connect if necessary
if (parsed.connect) {
config.parse_connect(parsed.connect, options);
}
if (options.project_name == null) {
throw new NiceError('No project_name given', {
description: `\
The project_name is needed to migrate from the v1.x format the v.2.0 format. \
It wasn't passed on the command line or found in your config.`,
suggestions: [
'pass the --project-name option to hz migrate',
'add the "project_name" key to your .hz/config.toml',
] });
}
return options;
}
function setup() {
// Start rethinkdb server if necessary
// Connect to whatever rethinkdb server we're using
white('Setup');
return Promise.resolve().then(() => {
if (this.options.project_path && this.options.project_path !== '.') {
green(` ├── Changing to directory ${this.options.project_path}`);
change_to_project_dir(this.options.project_path);
}
}).then(() => {
// start rethinkdb server if necessary
if (this.options.start_rethinkdb) {
green(' ├── Starting RethinkDB server');
return start_rdb_server({ quiet: true }).then((server) => {
this.rdb_server = server;
this.options.rdb_host = 'localhost';
this.options.rdb_port = server.driver_port;
});
}
}).then(() => {
green(' ├── Connecting to RethinkDB');
return r.connect({
host: this.options.rdb_host,
port: this.options.rdb_port,
user: this.options.rdb_user,
password: this.options.rdb_password,
});
}).then((conn) => {
green(' └── Successfully connected');
this.conn = conn;
});
}
function teardown() {
return Promise.resolve().then(() => {
white('Cleaning up...');
// close the rethinkdb connection
if (this.conn) {
green(' ├── Closing rethinkdb connection');
return this.conn.close();
}
}).then(() => {
// shut down the rethinkdb server if we started it
if (this.rdb_server) {
green(' └── Shutting down rethinkdb server');
return this.rdb_server.close();
}
});
}
function validateMigration() {
// check that `${project}_internal` exists
const project = this.options.project_name;
const internalNotFound = `Database named '${project}_internal' wasn't found`;
const tablesHaveHzPrefix = `Some tables in ${project} have an hz_ prefix`;
const checkForHzTables = r.db('rethinkdb')
.table('table_config')
.filter({ db: project })('name')
.contains((x) => x.match('^hz_'))
.branch(r.error(tablesHaveHzPrefix), true);
const waitForCollections = r.db(`${project}_internal`)
.table('collections')
.wait({ timeout: 30 })
.do(() => r.db(project).tableList())
.forEach((tableName) =>
r.db(project).table(tableName).wait({ timeout: 30 })
);
return Promise.resolve().then(() => {
white('Validating current schema version');
return r.dbList().contains(`${project}_internal`)
.branch(true, r.error(internalNotFound))
.do(() => checkForHzTables)
.do(() => waitForCollections)
.run(this.conn)
.then(() => green(' └── Pre-2.0 schema found'))
.catch((e) => {
if (e.msg === internalNotFound) {
throw new NiceError(e.msg, {
description: `\
This could happen if you don't have a Horizon app in this database, or if \
you've already migrated this database to the v2.0 format.`,
});
} else if (e.msg === tablesHaveHzPrefix) {
throw new NiceError(e.msg, {
description: `This could happen if you've already migrated \
this database to the v2.0 format.`,
});
} else {
throw e;
}
});
});
}
function makeBackup() {
// shell out to rethinkdb dump
const rdbHost = this.options.rdb_host;
const rdbPort = this.options.rdb_port;
if (this.options.skip_backup) {
return Promise.resolve();
}
white('Backing up rethinkdb_data directory');
if (this.options.nonportable_backup) {
return nonportableBackup();
}
return procPromise('rethinkdb', [
'dump',
'--connect',
`${rdbHost}:${rdbPort}`,
]).then(() => {
green(' └── Backup completed');
}).catch((e) => {
if (e.message.match(/Python driver/)) {
throw new NiceError('The RethinkDB Python driver is not installed.', {
description: `Before we migrate to the v2.0 format, we should do a \
backup of your RethinkDB database in case anything goes wrong. Unfortunately, \
we can't use the rethinkdb dump command to do a backup because you don't have \
the RethinkDB Python driver installed on your system.`,
suggestions: [
`Install the Python driver with the instructions found at: \
http://www.rethinkdb.com/docs/install-drivers/python/`,
`Pass the --nonportable-backup flag to hz migrate. This flag uses \
the tar command to make a backup, but the backup is not safe to use on \
another machine or to create replicas from. This option should not be used \
if RethinkDB is currently running. It should also not be used if the \
rethinkdb_data/ directory is not in the current directory.`,
] });
} else {
throw e;
}
});
}
function nonportableBackup() {
// Uses tar to do an unsafe backup
const timestamp = new Date().toISOString().replace(/:/g, '_');
return procPromise('tar', [
'-zcvf', // gzip, compress, verbose, filename is...
`rethinkdb_data.nonportable-backup.${timestamp}.tar.gz`,
'rethinkdb_data', // directory to back up
]).then(() => {
green(' └── Nonportable backup completed');
});
}
function renameUserTables() {
// for each table listed in ${project}_internal.collections
// rename the table name to the collection name
const project = this.options.project_name;
return Promise.resolve().then(() => {
white('Removing suffix from user tables');
return r.db(`${project}_internal`).wait({ timeout: 30 }).
do(() => r.db(`${project}_internal`).table('collections')
.forEach((collDoc) => r.db('rethinkdb').table('table_config')
.filter({ db: project, name: collDoc('table') })
.update({ name: collDoc('id') }))
).run(this.conn)
.then(() => green(' └── Suffixes removed'));
});
}
function moveInternalTables() {
// find project_internal
// move all tables from ${project}_internal.${table} to ${project}.hz_${table}
// - except for users, don't add hz_prefix, but move its db
const project = this.options.project_name;
return Promise.resolve().then(() => {
white(`Moving internal tables from ${project}_internal to ${project}`);
return r.db('rethinkdb').table('table_config')
.filter({ db: `${project}_internal` })
.update((table) => ({
db: project,
name: r.branch(
table('name').ne('users'),
r('hz_').add(table('name')),
'users'),
})).run(this.conn)
.then(() => green(' ├── Internal tables moved'));
}).then(() => {
// delete project_internal
green(` └── Deleting empty "${project}_internal" database`);
return r.dbDrop(`${project}_internal`).run(this.conn);
});
}
function renameIndices() {
// for each user $table in ${project}
// for each index in ${table}
// parse the old name into array of field names.
// rename to `hz_${JSON.stringify(fields)}`
const project = this.options.project_name;
return Promise.resolve().then(() => {
white('Renaming indices to new JSON format');
return r.db(project).tableList().forEach((tableName) =>
r.db(project).table(tableName).indexList().forEach((indexName) =>
r.db(project).table(tableName)
.indexRename(indexName, rename(indexName))
)
).run(this.conn)
.then(() => green(' └── Indices renamed.'));
});
function rename(name) {
// ReQL to rename the index name to the new format
const initialState = {
escaped: false,
field: '',
fields: [ ],
};
return name.split('')
.fold(initialState, (acc, c) =>
r.branch(
acc('escaped'),
acc.merge({
escaped: false,
field: acc('field').add(c),
}),
c.eq('\\'),
acc.merge({ escaped: true }),
c.eq('_'),
acc.merge({
fields: acc('fields').append(acc('field')),
field: '',
}),
acc.merge({ field: acc('field').add(c) })
)
).do((state) =>
// last field needs to be appended to running list
state('fields').append(state('field'))
// wrap each field in an array
.map((field) => [ field ])
)
.toJSON()
.do((x) => r('hz_').add(x));
}
}
function rewriteHzCollectionDocs() {
// for each document in ${project}.hz_collections
// delete the table field
const project = this.options.project_name;
return Promise.resolve().then(() => {
white('Rewriting hz_collections to new format');
return r.db(project).table('hz_collections')
.update({ table: r.literal() })
.run(this.conn);
}).then(() => green(' ├── "table" field removed'))
.then(() => r.db(project).table('hz_collections')
.insert({ id: 'users' })
.run(this.conn))
.then(() => green(' ├── Added document for "users" table'))
.then(() => r.db(project).table('hz_collections')
.insert({ id: 'hz_metadata', version: VERSION_2_0 })
.run(this.conn))
.then(() => green(' └── Adding the metadata document with schema version:' +
`${JSON.stringify(VERSION_2_0)}`));
}
function exportNewSchema() {
// Import and run schema save process, giving it a different
// filename than schema.toml
const timestamp = new Date().toISOString().replace(/:/g, '_');
return accessAsync('.hz/schema.toml', fs.R_OK | fs.F_OK)
.then(() => `.hz/schema.toml.migrated.${timestamp}`)
.catch(() => '.hz/schema.toml') // if no schema.toml
.then((schemaFile) => {
white(`Exporting the new schema to ${schemaFile}`);
return runSaveCommand({
rdb_host: this.options.rdb_host,
rdb_port: this.options.rdb_port,
rdb_user: this.options.rdb_user,
rdb_password: this.options.rdb_password,
out_file: schemaFile,
project_name: this.options.project_name,
});
}).then(() => green(' └── Schema exported'));
}
module.exports = {
run,
description: 'migrate an older version of horizon to a newer one',
};
================================================
FILE: cli/src/schema.js
================================================
'use strict';
const horizon_server = require('@horizon/server');
const horizon_index = require('@horizon/server/src/metadata/index');
const horizon_metadata = require('@horizon/server/src/metadata/metadata');
const config = require('./utils/config');
const interrupt = require('./utils/interrupt');
const start_rdb_server = require('./utils/start_rdb_server');
const parse_yes_no_option = require('./utils/parse_yes_no_option');
const change_to_project_dir = require('./utils/change_to_project_dir');
const initialize_joi = require('./utils/initialize_joi');
const fs = require('fs');
const Joi = require('joi');
const path = require('path');
const argparse = require('argparse');
const toml = require('toml');
const r = horizon_server.r;
const create_collection = horizon_metadata.create_collection;
const initialize_metadata = horizon_metadata.initialize_metadata;
initialize_joi(Joi);
const parseArguments = (args) => {
const parser = new argparse.ArgumentParser({ prog: 'hz schema' });
const subparsers = parser.addSubparsers({
title: 'subcommands',
dest: 'subcommand_name',
});
const apply = subparsers.addParser('apply', { addHelp: true });
const save = subparsers.addParser('save', { addHelp: true });
// Set options shared between both subcommands
[ apply, save ].map((subcmd) => {
subcmd.addArgument([ 'project_path' ],
{ type: 'string', nargs: '?',
help: 'Change to this directory before serving' });
subcmd.addArgument([ '--project-name', '-n' ],
{ type: 'string', action: 'store', metavar: 'NAME',
help: 'Name of the Horizon Project server' });
subcmd.addArgument([ '--connect', '-c' ],
{ type: 'string', metavar: 'HOST:PORT',
help: 'Host and port of the RethinkDB server to connect to.' });
subcmd.addArgument([ '--rdb-timeout' ],
{ type: 'int', metavar: 'TIMEOUT',
help: 'Timeout period in seconds for the RethinkDB connection to be opened' });
subcmd.addArgument([ '--rdb-user' ],
{ type: 'string', metavar: 'USER',
help: 'RethinkDB User' });
subcmd.addArgument([ '--rdb-password' ],
{ type: 'string', metavar: 'PASSWORD',
help: 'RethinkDB Password' });
subcmd.addArgument([ '--start-rethinkdb' ],
{ type: 'string', metavar: 'yes|no', constant: 'yes', nargs: '?',
help: 'Start up a RethinkDB server in the current directory' });
subcmd.addArgument([ '--debug' ],
{ type: 'string', metavar: 'yes|no', constant: 'yes', nargs: '?',
help: 'Enable debug logging.' });
});
// Options exclusive to HZ SCHEMA APPLY
apply.addArgument([ '--update' ],
{ type: 'string', metavar: 'yes|no', constant: 'yes', nargs: '?',
help: 'Only add new items and update existing, no removal.' });
apply.addArgument([ '--force' ],
{ type: 'string', metavar: 'yes|no', constant: 'yes', nargs: '?',
help: 'Allow removal of existing collections.' });
apply.addArgument([ 'schema_file' ],
{ type: 'string', metavar: 'SCHEMA_FILE_PATH',
help: 'File to get the horizon schema from, use "-" for stdin.' });
// Options exclusive to HZ SCHEMA SAVE
save.addArgument([ '--out-file', '-o' ],
{ type: 'string', metavar: 'PATH', defaultValue: '.hz/schema.toml',
help: 'File to write the horizon schema to, defaults to .hz/schema.toml.' });
return parser.parseArgs(args);
};
const schema_schema = Joi.object().unknown(false).keys({
collections: Joi.object().unknown(true).pattern(/.*/,
Joi.object().unknown(false).keys({
indexes: Joi.array().items(
Joi.alternatives(
Joi.string(),
Joi.object().unknown(false).keys({
fields: Joi.array().items(Joi.array().items(Joi.string())).required(),
})
)
).optional().default([ ]),
})
).optional().default({ }),
groups: Joi.object().unknown(true).pattern(/.*/,
Joi.object().keys({
rules: Joi.object().unknown(true).pattern(/.*/,
Joi.object().unknown(false).keys({
template: Joi.string().required(),
validator: Joi.string().optional(),
})
).optional().default({ }),
})
).optional().default({ }),
});
// Preserved for interpreting old schemas
const v1_0_name_to_fields = (name) => {
let escaped = false;
let field = '';
const fields = [ ];
for (const c of name) {
if (escaped) {
if (c !== '\\' && c !== '_') {
throw new Error(`Unexpected index name: "${name}"`);
}
escaped = false;
field += c;
} else if (c === '\\') {
escaped = true;
} else if (c === '_') {
fields.push(field);
field = '';
} else {
field += c;
}
}
if (escaped) {
throw new Error(`Unexpected index name: "${name}"`);
}
fields.push([ field ]);
return fields;
};
const parse_schema = (schema_toml) => {
const parsed = Joi.validate(toml.parse(schema_toml), schema_schema);
const schema = parsed.value;
if (parsed.error) {
throw parsed.error;
}
const collections = [ ];
for (const name in schema.collections) {
collections.push({
id: name,
indexes: schema.collections[name].indexes.map((index) => {
if (typeof index === 'string') {
return { fields: v1_0_name_to_fields(index), multi: false, geo: false };
} else {
return { fields: index.fields, multi: false, geo: false };
}
}),
});
}
// Make sure the 'users' collection is present, as some things depend on
// its existence.
if (!schema.collections || !schema.collections.users) {
collections.push({ id: 'users', indexes: [ ] });
}
const groups = [ ];
for (const name in schema.groups) {
groups.push(Object.assign({ id: name }, schema.groups[name]));
}
return { groups, collections };
};
const processApplyConfig = (parsed) => {
let options, in_file;
options = config.default_options();
options = config.merge_options(options,
config.read_from_config_file(parsed.project_path));
options = config.merge_options(options, config.read_from_env());
options = config.merge_options(options, config.read_from_flags(parsed));
if (parsed.schema_file === '-') {
in_file = process.stdin;
} else {
in_file = fs.createReadStream(parsed.schema_file, { flags: 'r' });
}
if (options.project_name === null) {
options.project_name = path.basename(path.resolve(options.project_path));
}
return {
subcommand_name: 'apply',
start_rethinkdb: options.start_rethinkdb,
rdb_host: options.rdb_host,
rdb_port: options.rdb_port,
rdb_user: options.rdb_user || undefined,
rdb_password: options.rdb_password || undefined,
project_name: options.project_name,
project_path: options.project_path,
debug: options.debug,
update: parse_yes_no_option(parsed.update),
force: parse_yes_no_option(parsed.force),
in_file,
};
};
const processSaveConfig = (parsed) => {
let options, out_file;
options = config.default_options();
options.start_rethinkdb = true;
options = config.merge_options(options,
config.read_from_config_file(parsed.project_path));
options = config.merge_options(options, config.read_from_env());
options = config.merge_options(options, config.read_from_flags(parsed));
if (parsed.out_file === '-') {
out_file = process.stdout;
} else {
out_file = parsed.out_file;
}
if (options.project_name === null) {
options.project_name = path.basename(path.resolve(options.project_path));
}
return {
subcommand_name: 'save',
start_rethinkdb: options.start_rethinkdb,
rdb_host: options.rdb_host,
rdb_port: options.rdb_port,
rdb_user: options.rdb_user || undefined,
rdb_password: options.rdb_password || undefined,
project_name: options.project_name,
project_path: options.project_path,
debug: options.debug,
out_file,
};
};
const schema_to_toml = (collections, groups) => {
const res = [ '# This is a TOML document' ];
for (const c of collections) {
res.push('');
res.push(`[collections.${c.id}]`);
c.indexes.forEach((index) => {
const info = horizon_index.name_to_info(index);
res.push(`[[collections.${c.id}.indexes]]`);
res.push(`fields = ${JSON.stringify(info.fields)}`);
});
}
for (const g of groups) {
res.push('');
res.push(`[groups.${g.id}]`);
if (g.rules) {
for (const key in g.rules) {
const template = g.rules[key].template;
const validator = g.rules[key].validator;
res.push(`[groups.${g.id}.rules.${key}]`);
res.push(`template = ${JSON.stringify(template)}`);
if (validator) {
res.push(`validator = ${JSON.stringify(validator)}`);
}
}
}
}
res.push('');
return res.join('\n');
};
const runApplyCommand = (options) => {
let conn, schema, rdb_server;
let obsolete_collections = [ ];
const db = options.project_name;
const cleanup = () =>
Promise.all([
conn ? conn.close() : Promise.resolve(),
rdb_server ? rdb_server.close() : Promise.resolve(),
]);
interrupt.on_interrupt(() => cleanup());
return Promise.resolve().then(() => {
if (options.start_rethinkdb) {
change_to_project_dir(options.project_path);
}
return new Promise((resolve, reject) => {
let schema_toml = '';
options.in_file.on('data', (buffer) => (schema_toml += buffer));
options.in_file.on('end', () => resolve(schema_toml));
options.in_file.on('error', reject);
});
}).then((schema_toml) => {
schema = parse_schema(schema_toml);
if (options.start_rethinkdb) {
return start_rdb_server({ quiet: !options.debug }).then((server) => {
rdb_server = server;
options.rdb_host = 'localhost';
options.rdb_port = server.driver_port;
});
}
}).then(() =>
r.connect({ host: options.rdb_host,
port: options.rdb_port,
user: options.rdb_user,
password: options.rdb_password,
timeout: options.rdb_timeout })
).then((rdb_conn) => {
conn = rdb_conn;
return initialize_metadata(db, conn);
}).then((initialization_result) => {
if (initialization_result.tables_created) {
console.log('Initialized new application metadata.');
}
// Wait for metadata tables to be writable
return r.expr([ 'hz_collections', 'hz_groups' ])
.forEach((table) =>
r.db(db).table(table)
.wait({ waitFor: 'ready_for_writes', timeout: 30 }))
.run(conn);
}).then(() => {
// Error if any collections will be removed
if (!options.update) {
return r.db(db).table('hz_collections')
.filter((row) => row('id').match('^hz_').not())
.getField('id')
.coerceTo('array')
.setDifference(schema.collections.map((c) => c.id))
.run(conn)
.then((res) => {
if (!options.force && res.length > 0) {
throw new Error('Run with "--force" to continue.\n' +
'These collections would be removed along with their data:\n' +
`${res.join(', ')}`);
}
obsolete_collections = res;
});
}
}).then(() => {
if (options.update) {
// Update groups
return Promise.all(schema.groups.map((group) => {
const literal_group = JSON.parse(JSON.stringify(group));
Object.keys(literal_group.rules).forEach((key) => {
literal_group.rules[key] = r.literal(literal_group.rules[key]);
});
return r.db(db).table('hz_groups')
.get(group.id).replace((old_row) =>
r.branch(old_row.eq(null),
group,
old_row.merge(literal_group)))
.run(conn).then((res) => {
if (res.errors) {
throw new Error(`Failed to update group: ${res.first_error}`);
}
});
}));
} else {
// Replace and remove groups
const groups_obj = { };
schema.groups.forEach((g) => { groups_obj[g.id] = g; });
return Promise.all([
r.expr(groups_obj).do((groups) =>
r.db(db).table('hz_groups')
.replace((old_row) =>
r.branch(groups.hasFields(old_row('id')),
old_row,
null))
).run(conn).then((res) => {
if (res.errors) {
throw new Error(`Failed to write groups: ${res.first_error}`);
}
}),
r.db(db).table('hz_groups')
.insert(schema.groups, { conflict: 'replace' })
.run(conn).then((res) => {
if (res.errors) {
throw new Error(`Failed to write groups: ${res.first_error}`);
}
}),
]);
}
}).then(() => {
// Ensure all collections exist and remove any obsolete collections
const promises = [ ];
for (const c of schema.collections) {
promises.push(
create_collection(db, c.id, conn).then((res) => {
if (res.error) {
throw new Error(res.error);
}
}));
}
for (const c of obsolete_collections) {
promises.push(
r.db(db)
.table('hz_collections')
.get(c)
.delete({ returnChanges: 'always' })('changes')(0)
.do((res) =>
r.branch(res.hasFields('error'),
res,
res('old_val').eq(null),
res,
r.db(db).tableDrop(res('old_val')('id')).do(() => res)))
.run(conn).then((res) => {
if (res.error) {
throw new Error(res.error);
}
}));
}
return Promise.all(promises);
}).then(() => {
const promises = [ ];
// Ensure all indexes exist
for (const c of schema.collections) {
for (const info of c.indexes) {
const name = horizon_index.info_to_name(info);
promises.push(
r.branch(r.db(db).table(c.id).indexList().contains(name), { },
r.db(db).table(c.id).indexCreate(name, horizon_index.info_to_reql(info),
{ geo: Boolean(info.geo), multi: (info.multi !== false) }))
.run(conn)
.then((res) => {
if (res.errors) {
throw new Error(`Failed to create index ${name} ` +
`on collection ${c.id}: ${res.first_error}`);
}
}));
}
}
// Remove obsolete indexes
if (!options.update) {
for (const c of schema.collections) {
const names = c.indexes.map(horizon_index.info_to_name);
promises.push(
r.db(db).table(c.id).indexList().filter((name) => name.match('^hz_'))
.setDifference(names)
.forEach((name) => r.db(db).table(c.id).indexDrop(name))
.run(conn)
.then((res) => {
if (res.errors) {
throw new Error('Failed to remove old indexes ' +
`on collection ${c.id}: ${res.first_error}`);
}
}));
}
}
return Promise.all(promises);
}).then(cleanup).catch((err) => cleanup().then(() => { throw err; }));
};
const file_exists = (filename) => {
try {
fs.accessSync(filename);
} catch (e) {
return false;
}
return true;
};
const runSaveCommand = (options) => {
let conn, rdb_server;
const db = options.project_name;
const cleanup = () =>
Promise.all([
conn ? conn.close() : Promise.resolve(),
rdb_server ? rdb_server.close() : Promise.resolve(),
]);
interrupt.on_interrupt(() => cleanup());
return Promise.resolve().then(() => {
if (options.start_rethinkdb) {
change_to_project_dir(options.project_path);
}
}).then(() => {
if (options.start_rethinkdb) {
return start_rdb_server({ quiet: !options.debug }).then((server) => {
rdb_server = server;
options.rdb_host = 'localhost';
options.rdb_port = server.driver_port;
});
}
}).then(() =>
r.connect({ host: options.rdb_host,
port: options.rdb_port,
user: options.rdb_user,
password: options.rdb_password,
timeout: options.rdb_timeout })
).then((rdb_conn) => {
conn = rdb_conn;
return r.db(db).wait({ waitFor: 'ready_for_reads', timeout: 30 }).run(conn);
}).then(() =>
r.object('collections',
r.db(db).table('hz_collections')
.filter((row) => row('id').match('^hz_').not())
.coerceTo('array')
.map((row) =>
row.merge({ indexes: r.db(db).table(row('id')).indexList() })),
'groups', r.db(db).table('hz_groups').coerceTo('array'))
.run(conn)
).then((res) =>
new Promise((resolve) => {
// Only rename old file if saving to default .hz/schema.toml
if (options.out_file === '.hz/schema.toml' &&
file_exists(options.out_file)) {
// Rename existing file to have the current time appended to its name
const oldPath = path.resolve(options.out_file);
const newPath = `${path.resolve(options.out_file)}.${new Date().toISOString()}`;
fs.renameSync(oldPath, newPath);
}
const output = (options.out_file === '-') ? process.stdout :
fs.createWriteStream(options.out_file, { flags: 'w', defaultEncoding: 'utf8' });
// Output toml_str to schema.toml
const toml_str = schema_to_toml(res.collections, res.groups);
output.end(toml_str, resolve);
})
).then(cleanup).catch((err) => cleanup().then(() => { throw err; }));
};
const processConfig = (options) => {
// Determine if we are saving or applying and use appropriate config processing
switch (options.subcommand_name) {
case 'apply':
return processApplyConfig(options);
case 'save':
return processSaveConfig(options);
default:
throw new Error(`Unrecognized schema subcommand: "${options.subcommand_name}"`);
}
};
// Avoiding cyclical depdendencies
module.exports = {
run: (args) =>
Promise.resolve().then(() => {
const options = processConfig(parseArguments(args));
// Determine if we are saving or applying and use appropriate run function
switch (options.subcommand_name) {
case 'apply':
return runApplyCommand(options);
case 'save':
return runSaveCommand(options);
default:
throw new Error(`Unrecognized schema subcommand: "${options.subcommand_name}"`);
}
}),
description: 'Apply and save the schema from a horizon database',
processApplyConfig,
runApplyCommand,
runSaveCommand,
parse_schema,
};
================================================
FILE: cli/src/serve.js
================================================
'use strict';
const chalk = require('chalk');
const crypto = require('crypto');
const fs = require('fs');
const get_type = require('mime-types').contentType;
const http = require('http');
const https = require('https');
const open = require('open');
const path = require('path');
const argparse = require('argparse');
const url = require('url');
const config = require('./utils/config');
const start_rdb_server = require('./utils/start_rdb_server');
const change_to_project_dir = require('./utils/change_to_project_dir');
const NiceError = require('./utils/nice_error.js');
const interrupt = require('./utils/interrupt');
const schema = require('./schema');
const horizon_server = require('@horizon/server');
const logger = horizon_server.logger;
const TIMEOUT_30_SECONDS = 30 * 1000;
const default_rdb_host = 'localhost';
const default_rdb_port = 28015;
const default_rdb_timeout = 20;
const parseArguments = (args) => {
const parser = new argparse.ArgumentParser({ prog: 'hz serve' });
parser.addArgument([ 'project_path' ],
{ type: 'string', nargs: '?',
help: 'Change to this directory before serving' });
parser.addArgument([ '--project-name', '-n' ],
{ type: 'string', action: 'store', metavar: 'NAME',
help: 'Name of the Horizon project. Determines the name of ' +
'the RethinkDB database that stores the project data.' });
parser.addArgument([ '--bind', '-b' ],
{ type: 'string', action: 'append', metavar: 'HOST',
help: 'Local hostname to serve horizon on (repeatable).' });
parser.addArgument([ '--port', '-p' ],
{ type: 'int', metavar: 'PORT',
help: 'Local port to serve horizon on.' });
parser.addArgument([ '--connect', '-c' ],
{ type: 'string', metavar: 'HOST:PORT',
help: 'Host and port of the RethinkDB server to connect to.' });
parser.addArgument([ '--rdb-timeout' ],
{ type: 'int', metavar: 'TIMEOUT',
help: 'Timeout period in seconds for the RethinkDB connection to be opened' });
parser.addArgument([ '--rdb-user' ],
{ type: 'string', metavar: 'USER',
help: 'RethinkDB User' });
parser.addArgument([ '--rdb-password' ],
{ type: 'string', metavar: 'PASSWORD',
help: 'RethinkDB Password' });
parser.addArgument([ '--key-file' ],
{ type: 'string', metavar: 'PATH',
help: 'Path to the key file to use, defaults to "./horizon-key.pem".' });
parser.addArgument([ '--cert-file' ],
{ type: 'string', metavar: 'PATH',
help: 'Path to the cert file to use, defaults to "./horizon-cert.pem".' });
parser.addArgument([ '--token-secret' ],
{ type: 'string', metavar: 'SECRET',
help: 'Key for signing jwts' });
parser.addArgument([ '--allow-unauthenticated' ],
{ type: 'string', metavar: 'yes|no', constant: 'yes', nargs: '?',
help: 'Whether to allow unauthenticated Horizon connections.' });
parser.addArgument([ '--allow-anonymous' ],
{ type: 'string', metavar: 'yes|no', constant: 'yes', nargs: '?',
help: 'Whether to allow anonymous Horizon connections.' });
parser.addArgument([ '--max-connections' ],
{ type: 'int', metavar: 'MAX_CONNECTIONS',
help: 'Maximum number of simultaneous connections server will accept.' });
parser.addArgument([ '--debug' ],
{ type: 'string', metavar: 'yes|no', constant: 'yes', nargs: '?',
help: 'Enable debug logging.' });
parser.addArgument([ '--secure' ],
{ type: 'string', metavar: 'yes|no', constant: 'yes', nargs: '?',
help: 'Serve secure websockets, requires --key-file and ' +
'--cert-file if true, on by default.' });
parser.addArgument([ '--start-rethinkdb' ],
{ type: 'string', metavar: 'yes|no', constant: 'yes', nargs: '?',
help: 'Start up a RethinkDB server in the current directory' });
parser.addArgument([ '--auto-create-collection' ],
{ type: 'string', metavar: 'yes|no', constant: 'yes', nargs: '?',
help: 'Create collections used by requests if they do not exist.' });
parser.addArgument([ '--auto-create-index' ],
{ type: 'string', metavar: 'yes|no', constant: 'yes', nargs: '?',
help: 'Create indexes used by requests if they do not exist.' });
parser.addArgument([ '--permissions' ],
{ type: 'string', metavar: 'yes|no', constant: 'yes', nargs: '?',
help: 'Enables or disables checking permissions on requests.' });
parser.addArgument([ '--serve-static' ],
{ type: 'string', metavar: 'PATH', nargs: '?', constant: './dist',
help: 'Serve static files from a directory, defaults to "./dist".' });
parser.addArgument([ '--dev' ],
{ action: 'storeTrue',
help: 'Runs the server in development mode, this sets ' +
'--secure=no, ' +
'--permissions=no, ' +
'--auto-create-collection=yes, ' +
'--auto-create-index=yes, ' +
'--start-rethinkdb=yes, ' +
'--allow-unauthenticated=yes, ' +
'--allow-anonymous=yes ' +
'and --serve-static=./dist.' });
parser.addArgument([ '--schema-file' ],
{ type: 'string', metavar: 'SCHEMA_FILE_PATH',
help: 'Path to the schema file to use, ' +
'will attempt to apply schema before starting Horizon server".' });
parser.addArgument([ '--auth' ],
{ type: 'string', action: 'append', metavar: 'PROVIDER,ID,SECRET', defaultValue: [ ],
help: 'Auth provider and options comma-separated, e.g. "facebook,todos
Authentication successful
Your user ID is ${user.id}
This small example will help you get started with setting up OAuth on Horizon. Here's what you need to get OAuth going:
- Create an application at Github.
- Add the client id and client secret to the configuration file
- Ensure that you can find Github in the registered providers
- Click the authenticate with Github button
- Connect to Horizon
- Configure Github OAuth
-
{messages}
{this.props.message.text}
-
{todoItems}
todos
-
{ text } { datetime.toTimeString() }
-
{{ message.text }} {{message.datetime.toTimeString()}}