Showing preview only (1,105K chars total). Download the full file or copy to clipboard to get everything.
Repository: louischatriot/nedb
Branch: master
Commit: 35be491ca67d
Files: 55
Total size: 1.0 MB
Directory structure:
gitextract_fo51vwzk/
├── .gitignore
├── LICENSE
├── README.md
├── benchmarks/
│ ├── commonUtilities.js
│ ├── ensureIndex.js
│ ├── find.js
│ ├── findOne.js
│ ├── findWithIn.js
│ ├── insert.js
│ ├── loadDatabase.js
│ ├── remove.js
│ └── update.js
├── bower.json
├── browser-version/
│ ├── browser-specific/
│ │ └── lib/
│ │ ├── customUtils.js
│ │ └── storage.js
│ ├── build.js
│ ├── out/
│ │ └── nedb.js
│ ├── package.json
│ └── test/
│ ├── async.js
│ ├── chai.js
│ ├── index.html
│ ├── localforage.js
│ ├── mocha.css
│ ├── mocha.js
│ ├── nedb-browser.js
│ ├── playground.html
│ ├── testLoad.html
│ ├── testLoad.js
│ ├── testPersistence.html
│ ├── testPersistence.js
│ ├── testPersistence2.html
│ └── testPersistence2.js
├── index.js
├── lib/
│ ├── cursor.js
│ ├── customUtils.js
│ ├── datastore.js
│ ├── executor.js
│ ├── indexes.js
│ ├── model.js
│ ├── persistence.js
│ └── storage.js
├── package.json
├── test/
│ ├── cursor.test.js
│ ├── customUtil.test.js
│ ├── db.test.js
│ ├── executor.test.js
│ ├── indexes.test.js
│ ├── mocha.opts
│ ├── model.test.js
│ └── persistence.test.js
└── test_lac/
├── loadAndCrash.test.js
├── openFds.test.js
├── openFdsLaunch.sh
├── openFdsTestFile
└── openFdsTestFile2
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
lib-cov
*.seed
*.log
*.csv
*.dat
*.out
*.pid
*.gz
.idea
pids
logs
results
npm-debug.log
workspace
node_modules
browser-version/src
browser-version/node_modules
*.swp
*~
*.swo
================================================
FILE: LICENSE
================================================
(The MIT License)
Copyright (c) 2013 Louis Chatriot <louis.chatriot@gmail.com>
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
'Software'), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
================================================
FILE: README.md
================================================
<img src="http://i.imgur.com/9O1xHFb.png" style="width: 25%; height: 25%; float: left;">
## The JavaScript Database
> :warning: :warning: :warning: **WARNING:** this library is no longer maintained, and may have bugs and security issues. Feel free to fork but no pull request or security alert will be answered.
**Embedded persistent or in memory database for Node.js, nw.js, Electron and browsers, 100% JavaScript, no binary dependency**. API is a subset of MongoDB's and it's <a href="#speed">plenty fast</a>.
**IMPORTANT NOTE**: Please don't submit issues for questions regarding your code. Only actual bugs or feature requests will be answered, all others will be closed without comment. Also, please follow the <a href="#bug-reporting-guidelines">bug reporting guidelines</a> and check the <a href="https://github.com/louischatriot/nedb/wiki/Change-log" target="_blank">change log</a> before submitting an already fixed bug :)
## Installation, tests
Module name on npm and bower is `nedb`.
```
npm install nedb --save # Put latest version in your package.json
npm test # You'll need the dev dependencies to launch tests
bower install nedb # For the browser versions, which will be in browser-version/out
```
## API
It is a subset of MongoDB's API (the most used operations).
* <a href="#creatingloading-a-database">Creating/loading a database</a>
* <a href="#persistence">Persistence</a>
* <a href="#inserting-documents">Inserting documents</a>
* <a href="#finding-documents">Finding documents</a>
* <a href="#basic-querying">Basic Querying</a>
* <a href="#operators-lt-lte-gt-gte-in-nin-ne-exists-regex">Operators ($lt, $lte, $gt, $gte, $in, $nin, $ne, $exists, $regex)</a>
* <a href="#array-fields">Array fields</a>
* <a href="#logical-operators-or-and-not-where">Logical operators $or, $and, $not, $where</a>
* <a href="#sorting-and-paginating">Sorting and paginating</a>
* <a href="#projections">Projections</a>
* <a href="#counting-documents">Counting documents</a>
* <a href="#updating-documents">Updating documents</a>
* <a href="#removing-documents">Removing documents</a>
* <a href="#indexing">Indexing</a>
* <a href="#browser-version">Browser version</a>
### Creating/loading a database
You can use NeDB as an in-memory only datastore or as a persistent datastore. One datastore is the equivalent of a MongoDB collection. The constructor is used as follows `new Datastore(options)` where `options` is an object with the following fields:
* `filename` (optional): path to the file where the data is persisted. If left blank, the datastore is automatically considered in-memory only. It cannot end with a `~` which is used in the temporary files NeDB uses to perform crash-safe writes.
* `inMemoryOnly` (optional, defaults to `false`): as the name implies.
* `timestampData` (optional, defaults to `false`): timestamp the insertion and last update of all documents, with the fields `createdAt` and `updatedAt`. User-specified values override automatic generation, usually useful for testing.
* `autoload` (optional, defaults to `false`): if used, the database will automatically be loaded from the datafile upon creation (you don't need to call `loadDatabase`). Any command issued before load is finished is buffered and will be executed when load is done.
* `onload` (optional): if you use autoloading, this is the handler called after the `loadDatabase`. It takes one `error` argument. If you use autoloading without specifying this handler, and an error happens during load, an error will be thrown.
* `afterSerialization` (optional): hook you can use to transform data after it was serialized and before it is written to disk. Can be used for example to encrypt data before writing database to disk. This function takes a string as parameter (one line of an NeDB data file) and outputs the transformed string, **which must absolutely not contain a `\n` character** (or data will be lost).
* `beforeDeserialization` (optional): inverse of `afterSerialization`. Make sure to include both and not just one or you risk data loss. For the same reason, make sure both functions are inverses of one another. Some failsafe mechanisms are in place to prevent data loss if you misuse the serialization hooks: NeDB checks that never one is declared without the other, and checks that they are reverse of one another by testing on random strings of various lengths. In addition, if too much data is detected as corrupt, NeDB will refuse to start as it could mean you're not using the deserialization hook corresponding to the serialization hook used before (see below).
* `corruptAlertThreshold` (optional): between 0 and 1, defaults to 10%. NeDB will refuse to start if more than this percentage of the datafile is corrupt. 0 means you don't tolerate any corruption, 1 means you don't care.
* `compareStrings` (optional): function compareStrings(a, b) compares
strings a and b and return -1, 0 or 1. If specified, it overrides
default string comparison which is not well adapted to non-US characters
in particular accented letters. Native `localCompare` will most of the
time be the right choice
* `nodeWebkitAppName` (optional, **DEPRECATED**): if you are using NeDB from whithin a Node Webkit app, specify its name (the same one you use in the `package.json`) in this field and the `filename` will be relative to the directory Node Webkit uses to store the rest of the application's data (local storage etc.). It works on Linux, OS X and Windows. Now that you can use `require('nw.gui').App.dataPath` in Node Webkit to get the path to the data directory for your application, you should not use this option anymore and it will be removed.
If you use a persistent datastore without the `autoload` option, you need to call `loadDatabase` manually.
This function fetches the data from datafile and prepares the database. **Don't forget it!** If you use a
persistent datastore, no command (insert, find, update, remove) will be executed before `loadDatabase`
is called, so make sure to call it yourself or use the `autoload` option.
Also, if `loadDatabase` fails, all commands registered to the executor afterwards will not be executed. They will be registered and executed, in sequence, only after a successful `loadDatabase`.
```javascript
// Type 1: In-memory only datastore (no need to load the database)
var Datastore = require('nedb')
, db = new Datastore();
// Type 2: Persistent datastore with manual loading
var Datastore = require('nedb')
, db = new Datastore({ filename: 'path/to/datafile' });
db.loadDatabase(function (err) { // Callback is optional
// Now commands will be executed
});
// Type 3: Persistent datastore with automatic loading
var Datastore = require('nedb')
, db = new Datastore({ filename: 'path/to/datafile', autoload: true });
// You can issue commands right away
// Type 4: Persistent datastore for a Node Webkit app called 'nwtest'
// For example on Linux, the datafile will be ~/.config/nwtest/nedb-data/something.db
var Datastore = require('nedb')
, path = require('path')
, db = new Datastore({ filename: path.join(require('nw.gui').App.dataPath, 'something.db') });
// Of course you can create multiple datastores if you need several
// collections. In this case it's usually a good idea to use autoload for all collections.
db = {};
db.users = new Datastore('path/to/users.db');
db.robots = new Datastore('path/to/robots.db');
// You need to load each database (here we do it asynchronously)
db.users.loadDatabase();
db.robots.loadDatabase();
```
### Persistence
Under the hood, NeDB's persistence uses an append-only format, meaning that all updates and deletes actually result in lines added at the end of the datafile, for performance reasons. The database is automatically compacted (i.e. put back in the one-line-per-document format) every time you load each database within your application.
You can manually call the compaction function with `yourDatabase.persistence.compactDatafile` which takes no argument. It queues a compaction of the datafile in the executor, to be executed sequentially after all pending operations. The datastore will fire a `compaction.done` event once compaction is finished.
You can also set automatic compaction at regular intervals with `yourDatabase.persistence.setAutocompactionInterval(interval)`, `interval` in milliseconds (a minimum of 5s is enforced), and stop automatic compaction with `yourDatabase.persistence.stopAutocompaction()`.
Keep in mind that compaction takes a bit of time (not too much: 130ms for 50k records on a typical development machine) and no other operation can happen when it does, so most projects actually don't need to use it.
Compaction will also immediately remove any documents whose data line has become corrupted, assuming that the total percentage of all corrupted documents in that database still falls below the specified `corruptAlertThreshold` option's value.
Durability works similarly to major databases: compaction forces the OS to physically flush data to disk, while appends to the data file do not (the OS is responsible for flushing the data). That guarantees that a server crash can never cause complete data loss, while preserving performance. The worst that can happen is a crash between two syncs, causing a loss of all data between the two syncs. Usually syncs are 30 seconds appart so that's at most 30 seconds of data. <a href="http://oldblog.antirez.com/post/redis-persistence-demystified.html" target="_blank">This post by Antirez on Redis persistence</a> explains this in more details, NeDB being very close to Redis AOF persistence with `appendfsync` option set to `no`.
### Inserting documents
The native types are `String`, `Number`, `Boolean`, `Date` and `null`. You can also use
arrays and subdocuments (objects). If a field is `undefined`, it will not be saved (this is different from
MongoDB which transforms `undefined` in `null`, something I find counter-intuitive).
If the document does not contain an `_id` field, NeDB will automatically generated one for you (a 16-characters alphanumerical string). The `_id` of a document, once set, cannot be modified.
Field names cannot begin by '$' or contain a '.'.
```javascript
var doc = { hello: 'world'
, n: 5
, today: new Date()
, nedbIsAwesome: true
, notthere: null
, notToBeSaved: undefined // Will not be saved
, fruits: [ 'apple', 'orange', 'pear' ]
, infos: { name: 'nedb' }
};
db.insert(doc, function (err, newDoc) { // Callback is optional
// newDoc is the newly inserted document, including its _id
// newDoc has no key called notToBeSaved since its value was undefined
});
```
You can also bulk-insert an array of documents. This operation is atomic, meaning that if one insert fails due to a unique constraint being violated, all changes are rolled back.
```javascript
db.insert([{ a: 5 }, { a: 42 }], function (err, newDocs) {
// Two documents were inserted in the database
// newDocs is an array with these documents, augmented with their _id
});
// If there is a unique constraint on field 'a', this will fail
db.insert([{ a: 5 }, { a: 42 }, { a: 5 }], function (err) {
// err is a 'uniqueViolated' error
// The database was not modified
});
```
### Finding documents
Use `find` to look for multiple documents matching you query, or `findOne` to look for one specific document. You can select documents based on field equality or use comparison operators (`$lt`, `$lte`, `$gt`, `$gte`, `$in`, `$nin`, `$ne`). You can also use logical operators `$or`, `$and`, `$not` and `$where`. See below for the syntax.
You can use regular expressions in two ways: in basic querying in place of a string, or with the `$regex` operator.
You can sort and paginate results using the cursor API (see below).
You can use standard projections to restrict the fields to appear in the results (see below).
#### Basic querying
Basic querying means are looking for documents whose fields match the ones you specify. You can use regular expression to match strings.
You can use the dot notation to navigate inside nested documents, arrays, arrays of subdocuments and to match a specific element of an array.
```javascript
// Let's say our datastore contains the following collection
// { _id: 'id1', planet: 'Mars', system: 'solar', inhabited: false, satellites: ['Phobos', 'Deimos'] }
// { _id: 'id2', planet: 'Earth', system: 'solar', inhabited: true, humans: { genders: 2, eyes: true } }
// { _id: 'id3', planet: 'Jupiter', system: 'solar', inhabited: false }
// { _id: 'id4', planet: 'Omicron Persei 8', system: 'futurama', inhabited: true, humans: { genders: 7 } }
// { _id: 'id5', completeData: { planets: [ { name: 'Earth', number: 3 }, { name: 'Mars', number: 2 }, { name: 'Pluton', number: 9 } ] } }
// Finding all planets in the solar system
db.find({ system: 'solar' }, function (err, docs) {
// docs is an array containing documents Mars, Earth, Jupiter
// If no document is found, docs is equal to []
});
// Finding all planets whose name contain the substring 'ar' using a regular expression
db.find({ planet: /ar/ }, function (err, docs) {
// docs contains Mars and Earth
});
// Finding all inhabited planets in the solar system
db.find({ system: 'solar', inhabited: true }, function (err, docs) {
// docs is an array containing document Earth only
});
// Use the dot-notation to match fields in subdocuments
db.find({ "humans.genders": 2 }, function (err, docs) {
// docs contains Earth
});
// Use the dot-notation to navigate arrays of subdocuments
db.find({ "completeData.planets.name": "Mars" }, function (err, docs) {
// docs contains document 5
});
db.find({ "completeData.planets.name": "Jupiter" }, function (err, docs) {
// docs is empty
});
db.find({ "completeData.planets.0.name": "Earth" }, function (err, docs) {
// docs contains document 5
// If we had tested against "Mars" docs would be empty because we are matching against a specific array element
});
// You can also deep-compare objects. Don't confuse this with dot-notation!
db.find({ humans: { genders: 2 } }, function (err, docs) {
// docs is empty, because { genders: 2 } is not equal to { genders: 2, eyes: true }
});
// Find all documents in the collection
db.find({}, function (err, docs) {
});
// The same rules apply when you want to only find one document
db.findOne({ _id: 'id1' }, function (err, doc) {
// doc is the document Mars
// If no document is found, doc is null
});
```
#### Operators ($lt, $lte, $gt, $gte, $in, $nin, $ne, $exists, $regex)
The syntax is `{ field: { $op: value } }` where `$op` is any comparison operator:
* `$lt`, `$lte`: less than, less than or equal
* `$gt`, `$gte`: greater than, greater than or equal
* `$in`: member of. `value` must be an array of values
* `$ne`, `$nin`: not equal, not a member of
* `$exists`: checks whether the document posses the property `field`. `value` should be true or false
* `$regex`: checks whether a string is matched by the regular expression. Contrary to MongoDB, the use of `$options` with `$regex` is not supported, because it doesn't give you more power than regex flags. Basic queries are more readable so only use the `$regex` operator when you need to use another operator with it (see example below)
```javascript
// $lt, $lte, $gt and $gte work on numbers and strings
db.find({ "humans.genders": { $gt: 5 } }, function (err, docs) {
// docs contains Omicron Persei 8, whose humans have more than 5 genders (7).
});
// When used with strings, lexicographical order is used
db.find({ planet: { $gt: 'Mercury' }}, function (err, docs) {
// docs contains Omicron Persei 8
})
// Using $in. $nin is used in the same way
db.find({ planet: { $in: ['Earth', 'Jupiter'] }}, function (err, docs) {
// docs contains Earth and Jupiter
});
// Using $exists
db.find({ satellites: { $exists: true } }, function (err, docs) {
// docs contains only Mars
});
// Using $regex with another operator
db.find({ planet: { $regex: /ar/, $nin: ['Jupiter', 'Earth'] } }, function (err, docs) {
// docs only contains Mars because Earth was excluded from the match by $nin
});
```
#### Array fields
When a field in a document is an array, NeDB first tries to see if the query value is an array to perform an exact match, then whether there is an array-specific comparison function (for now there is only `$size` and `$elemMatch`) being used. If not, the query is treated as a query on every element and there is a match if at least one element matches.
* `$size`: match on the size of the array
* `$elemMatch`: matches if at least one array element matches the query entirely
```javascript
// Exact match
db.find({ satellites: ['Phobos', 'Deimos'] }, function (err, docs) {
// docs contains Mars
})
db.find({ satellites: ['Deimos', 'Phobos'] }, function (err, docs) {
// docs is empty
})
// Using an array-specific comparison function
// $elemMatch operator will provide match for a document, if an element from the array field satisfies all the conditions specified with the `$elemMatch` operator
db.find({ completeData: { planets: { $elemMatch: { name: 'Earth', number: 3 } } } }, function (err, docs) {
// docs contains documents with id 5 (completeData)
});
db.find({ completeData: { planets: { $elemMatch: { name: 'Earth', number: 5 } } } }, function (err, docs) {
// docs is empty
});
// You can use inside #elemMatch query any known document query operator
db.find({ completeData: { planets: { $elemMatch: { name: 'Earth', number: { $gt: 2 } } } } }, function (err, docs) {
// docs contains documents with id 5 (completeData)
});
// Note: you can't use nested comparison functions, e.g. { $size: { $lt: 5 } } will throw an error
db.find({ satellites: { $size: 2 } }, function (err, docs) {
// docs contains Mars
});
db.find({ satellites: { $size: 1 } }, function (err, docs) {
// docs is empty
});
// If a document's field is an array, matching it means matching any element of the array
db.find({ satellites: 'Phobos' }, function (err, docs) {
// docs contains Mars. Result would have been the same if query had been { satellites: 'Deimos' }
});
// This also works for queries that use comparison operators
db.find({ satellites: { $lt: 'Amos' } }, function (err, docs) {
// docs is empty since Phobos and Deimos are after Amos in lexicographical order
});
// This also works with the $in and $nin operator
db.find({ satellites: { $in: ['Moon', 'Deimos'] } }, function (err, docs) {
// docs contains Mars (the Earth document is not complete!)
});
```
#### Logical operators $or, $and, $not, $where
You can combine queries using logical operators:
* For `$or` and `$and`, the syntax is `{ $op: [query1, query2, ...] }`.
* For `$not`, the syntax is `{ $not: query }`
* For `$where`, the syntax is `{ $where: function () { /* object is "this", return a boolean */ } }`
```javascript
db.find({ $or: [{ planet: 'Earth' }, { planet: 'Mars' }] }, function (err, docs) {
// docs contains Earth and Mars
});
db.find({ $not: { planet: 'Earth' } }, function (err, docs) {
// docs contains Mars, Jupiter, Omicron Persei 8
});
db.find({ $where: function () { return Object.keys(this) > 6; } }, function (err, docs) {
// docs with more than 6 properties
});
// You can mix normal queries, comparison queries and logical operators
db.find({ $or: [{ planet: 'Earth' }, { planet: 'Mars' }], inhabited: true }, function (err, docs) {
// docs contains Earth
});
```
#### Sorting and paginating
If you don't specify a callback to `find`, `findOne` or `count`, a `Cursor` object is returned. You can modify the cursor with `sort`, `skip` and `limit` and then execute it with `exec(callback)`.
```javascript
// Let's say the database contains these 4 documents
// doc1 = { _id: 'id1', planet: 'Mars', system: 'solar', inhabited: false, satellites: ['Phobos', 'Deimos'] }
// doc2 = { _id: 'id2', planet: 'Earth', system: 'solar', inhabited: true, humans: { genders: 2, eyes: true } }
// doc3 = { _id: 'id3', planet: 'Jupiter', system: 'solar', inhabited: false }
// doc4 = { _id: 'id4', planet: 'Omicron Persei 8', system: 'futurama', inhabited: true, humans: { genders: 7 } }
// No query used means all results are returned (before the Cursor modifiers)
db.find({}).sort({ planet: 1 }).skip(1).limit(2).exec(function (err, docs) {
// docs is [doc3, doc1]
});
// You can sort in reverse order like this
db.find({ system: 'solar' }).sort({ planet: -1 }).exec(function (err, docs) {
// docs is [doc1, doc3, doc2]
});
// You can sort on one field, then another, and so on like this:
db.find({}).sort({ firstField: 1, secondField: -1 }) ... // You understand how this works!
```
#### Projections
You can give `find` and `findOne` an optional second argument, `projections`. The syntax is the same as MongoDB: `{ a: 1, b: 1 }` to return only the `a` and `b` fields, `{ a: 0, b: 0 }` to omit these two fields. You cannot use both modes at the time, except for `_id` which is by default always returned and which you can choose to omit. You can project on nested documents.
```javascript
// Same database as above
// Keeping only the given fields
db.find({ planet: 'Mars' }, { planet: 1, system: 1 }, function (err, docs) {
// docs is [{ planet: 'Mars', system: 'solar', _id: 'id1' }]
});
// Keeping only the given fields but removing _id
db.find({ planet: 'Mars' }, { planet: 1, system: 1, _id: 0 }, function (err, docs) {
// docs is [{ planet: 'Mars', system: 'solar' }]
});
// Omitting only the given fields and removing _id
db.find({ planet: 'Mars' }, { planet: 0, system: 0, _id: 0 }, function (err, docs) {
// docs is [{ inhabited: false, satellites: ['Phobos', 'Deimos'] }]
});
// Failure: using both modes at the same time
db.find({ planet: 'Mars' }, { planet: 0, system: 1 }, function (err, docs) {
// err is the error message, docs is undefined
});
// You can also use it in a Cursor way but this syntax is not compatible with MongoDB
db.find({ planet: 'Mars' }).projection({ planet: 1, system: 1 }).exec(function (err, docs) {
// docs is [{ planet: 'Mars', system: 'solar', _id: 'id1' }]
});
// Project on a nested document
db.findOne({ planet: 'Earth' }).projection({ planet: 1, 'humans.genders': 1 }).exec(function (err, doc) {
// doc is { planet: 'Earth', _id: 'id2', humans: { genders: 2 } }
});
```
### Counting documents
You can use `count` to count documents. It has the same syntax as `find`. For example:
```javascript
// Count all planets in the solar system
db.count({ system: 'solar' }, function (err, count) {
// count equals to 3
});
// Count all documents in the datastore
db.count({}, function (err, count) {
// count equals to 4
});
```
### Updating documents
`db.update(query, update, options, callback)` will update all documents matching `query` according to the `update` rules:
* `query` is the same kind of finding query you use with `find` and `findOne`
* `update` specifies how the documents should be modified. It is either a new document or a set of modifiers (you cannot use both together, it doesn't make sense!)
* A new document will replace the matched docs
* The modifiers create the fields they need to modify if they don't exist, and you can apply them to subdocs. Available field modifiers are `$set` to change a field's value, `$unset` to delete a field, `$inc` to increment a field's value and `$min`/`$max` to change field's value, only if provided value is less/greater than current value. To work on arrays, you have `$push`, `$pop`, `$addToSet`, `$pull`, and the special `$each` and `$slice`. See examples below for the syntax.
* `options` is an object with two possible parameters
* `multi` (defaults to `false`) which allows the modification of several documents if set to true
* `upsert` (defaults to `false`) if you want to insert a new document corresponding to the `update` rules if your `query` doesn't match anything. If your `update` is a simple object with no modifiers, it is the inserted document. In the other case, the `query` is stripped from all operator recursively, and the `update` is applied to it.
* `returnUpdatedDocs` (defaults to `false`, not MongoDB-compatible) if set to true and update is not an upsert, will return the array of documents matched by the find query and updated. Updated documents will be returned even if the update did not actually modify them.
* `callback` (optional) signature: `(err, numAffected, affectedDocuments, upsert)`. **Warning**: the API was changed between v1.7.4 and v1.8. Please refer to the <a href="https://github.com/louischatriot/nedb/wiki/Change-log" target="_blank">change log</a> to see the change.
* For an upsert, `affectedDocuments` contains the inserted document and the `upsert` flag is set to `true`.
* For a standard update with `returnUpdatedDocs` flag set to `false`, `affectedDocuments` is not set.
* For a standard update with `returnUpdatedDocs` flag set to `true` and `multi` to `false`, `affectedDocuments` is the updated document.
* For a standard update with `returnUpdatedDocs` flag set to `true` and `multi` to `true`, `affectedDocuments` is the array of updated documents.
**Note**: you can't change a document's _id.
```javascript
// Let's use the same example collection as in the "finding document" part
// { _id: 'id1', planet: 'Mars', system: 'solar', inhabited: false }
// { _id: 'id2', planet: 'Earth', system: 'solar', inhabited: true }
// { _id: 'id3', planet: 'Jupiter', system: 'solar', inhabited: false }
// { _id: 'id4', planet: 'Omicron Persia 8', system: 'futurama', inhabited: true }
// Replace a document by another
db.update({ planet: 'Jupiter' }, { planet: 'Pluton'}, {}, function (err, numReplaced) {
// numReplaced = 1
// The doc #3 has been replaced by { _id: 'id3', planet: 'Pluton' }
// Note that the _id is kept unchanged, and the document has been replaced
// (the 'system' and inhabited fields are not here anymore)
});
// Set an existing field's value
db.update({ system: 'solar' }, { $set: { system: 'solar system' } }, { multi: true }, function (err, numReplaced) {
// numReplaced = 3
// Field 'system' on Mars, Earth, Jupiter now has value 'solar system'
});
// Setting the value of a non-existing field in a subdocument by using the dot-notation
db.update({ planet: 'Mars' }, { $set: { "data.satellites": 2, "data.red": true } }, {}, function () {
// Mars document now is { _id: 'id1', system: 'solar', inhabited: false
// , data: { satellites: 2, red: true }
// }
// Not that to set fields in subdocuments, you HAVE to use dot-notation
// Using object-notation will just replace the top-level field
db.update({ planet: 'Mars' }, { $set: { data: { satellites: 3 } } }, {}, function () {
// Mars document now is { _id: 'id1', system: 'solar', inhabited: false
// , data: { satellites: 3 }
// }
// You lost the "data.red" field which is probably not the intended behavior
});
});
// Deleting a field
db.update({ planet: 'Mars' }, { $unset: { planet: true } }, {}, function () {
// Now the document for Mars doesn't contain the planet field
// You can unset nested fields with the dot notation of course
});
// Upserting a document
db.update({ planet: 'Pluton' }, { planet: 'Pluton', inhabited: false }, { upsert: true }, function (err, numReplaced, upsert) {
// numReplaced = 1, upsert = { _id: 'id5', planet: 'Pluton', inhabited: false }
// A new document { _id: 'id5', planet: 'Pluton', inhabited: false } has been added to the collection
});
// If you upsert with a modifier, the upserted doc is the query modified by the modifier
// This is simpler than it sounds :)
db.update({ planet: 'Pluton' }, { $inc: { distance: 38 } }, { upsert: true }, function () {
// A new document { _id: 'id5', planet: 'Pluton', distance: 38 } has been added to the collection
});
// If we insert a new document { _id: 'id6', fruits: ['apple', 'orange', 'pear'] } in the collection,
// let's see how we can modify the array field atomically
// $push inserts new elements at the end of the array
db.update({ _id: 'id6' }, { $push: { fruits: 'banana' } }, {}, function () {
// Now the fruits array is ['apple', 'orange', 'pear', 'banana']
});
// $pop removes an element from the end (if used with 1) or the front (if used with -1) of the array
db.update({ _id: 'id6' }, { $pop: { fruits: 1 } }, {}, function () {
// Now the fruits array is ['apple', 'orange']
// With { $pop: { fruits: -1 } }, it would have been ['orange', 'pear']
});
// $addToSet adds an element to an array only if it isn't already in it
// Equality is deep-checked (i.e. $addToSet will not insert an object in an array already containing the same object)
// Note that it doesn't check whether the array contained duplicates before or not
db.update({ _id: 'id6' }, { $addToSet: { fruits: 'apple' } }, {}, function () {
// The fruits array didn't change
// If we had used a fruit not in the array, e.g. 'banana', it would have been added to the array
});
// $pull removes all values matching a value or even any NeDB query from the array
db.update({ _id: 'id6' }, { $pull: { fruits: 'apple' } }, {}, function () {
// Now the fruits array is ['orange', 'pear']
});
db.update({ _id: 'id6' }, { $pull: { fruits: $in: ['apple', 'pear'] } }, {}, function () {
// Now the fruits array is ['orange']
});
// $each can be used to $push or $addToSet multiple values at once
// This example works the same way with $addToSet
db.update({ _id: 'id6' }, { $push: { fruits: { $each: ['banana', 'orange'] } } }, {}, function () {
// Now the fruits array is ['apple', 'orange', 'pear', 'banana', 'orange']
});
// $slice can be used in cunjunction with $push and $each to limit the size of the resulting array.
// A value of 0 will update the array to an empty array. A positive value n will keep only the n first elements
// A negative value -n will keep only the last n elements.
// If $slice is specified but not $each, $each is set to []
db.update({ _id: 'id6' }, { $push: { fruits: { $each: ['banana'], $slice: 2 } } }, {}, function () {
// Now the fruits array is ['apple', 'orange']
});
// $min/$max to update only if provided value is less/greater than current value
// Let's say the database contains this document
// doc = { _id: 'id', name: 'Name', value: 5 }
db.update({ _id: 'id1' }, { $min: { value: 2 } }, {}, function () {
// The document will be updated to { _id: 'id', name: 'Name', value: 2 }
});
db.update({ _id: 'id1' }, { $min: { value: 8 } }, {}, function () {
// The document will not be modified
});
```
### Removing documents
`db.remove(query, options, callback)` will remove all documents matching `query` according to `options`
* `query` is the same as the ones used for finding and updating
* `options` only one option for now: `multi` which allows the removal of multiple documents if set to true. Default is false
* `callback` is optional, signature: err, numRemoved
```javascript
// Let's use the same example collection as in the "finding document" part
// { _id: 'id1', planet: 'Mars', system: 'solar', inhabited: false }
// { _id: 'id2', planet: 'Earth', system: 'solar', inhabited: true }
// { _id: 'id3', planet: 'Jupiter', system: 'solar', inhabited: false }
// { _id: 'id4', planet: 'Omicron Persia 8', system: 'futurama', inhabited: true }
// Remove one document from the collection
// options set to {} since the default for multi is false
db.remove({ _id: 'id2' }, {}, function (err, numRemoved) {
// numRemoved = 1
});
// Remove multiple documents
db.remove({ system: 'solar' }, { multi: true }, function (err, numRemoved) {
// numRemoved = 3
// All planets from the solar system were removed
});
// Removing all documents with the 'match-all' query
db.remove({}, { multi: true }, function (err, numRemoved) {
});
```
### Indexing
NeDB supports indexing. It gives a very nice speed boost and can be used to enforce a unique constraint on a field. You can index any field, including fields in nested documents using the dot notation. For now, indexes are only used to speed up basic queries and queries using `$in`, `$lt`, `$lte`, `$gt` and `$gte`. The indexed values cannot be of type array of object.
To create an index, use `datastore.ensureIndex(options, cb)`, where callback is optional and get passed an error if any (usually a unique constraint that was violated). `ensureIndex` can be called when you want, even after some data was inserted, though it's best to call it at application startup. The options are:
* **fieldName** (required): name of the field to index. Use the dot notation to index a field in a nested document.
* **unique** (optional, defaults to `false`): enforce field uniqueness. Note that a unique index will raise an error if you try to index two documents for which the field is not defined.
* **sparse** (optional, defaults to `false`): don't index documents for which the field is not defined. Use this option along with "unique" if you want to accept multiple documents for which it is not defined.
* **expireAfterSeconds** (number of seconds, optional): if set, the created index is a TTL (time to live) index, that will automatically remove documents when the system date becomes larger than the date on the indexed field plus `expireAfterSeconds`. Documents where the indexed field is not specified or not a `Date` object are ignored
Note: the `_id` is automatically indexed with a unique constraint, no need to call `ensureIndex` on it.
You can remove a previously created index with `datastore.removeIndex(fieldName, cb)`.
If your datastore is persistent, the indexes you created are persisted in the datafile, when you load the database a second time they are automatically created for you. No need to remove any `ensureIndex` though, if it is called on a database that already has the index, nothing happens.
```javascript
db.ensureIndex({ fieldName: 'somefield' }, function (err) {
// If there was an error, err is not null
});
// Using a unique constraint with the index
db.ensureIndex({ fieldName: 'somefield', unique: true }, function (err) {
});
// Using a sparse unique index
db.ensureIndex({ fieldName: 'somefield', unique: true, sparse: true }, function (err) {
});
// Format of the error message when the unique constraint is not met
db.insert({ somefield: 'nedb' }, function (err) {
// err is null
db.insert({ somefield: 'nedb' }, function (err) {
// err is { errorType: 'uniqueViolated'
// , key: 'name'
// , message: 'Unique constraint violated for key name' }
});
});
// Remove index on field somefield
db.removeIndex('somefield', function (err) {
});
// Example of using expireAfterSeconds to remove documents 1 hour
// after their creation (db's timestampData option is true here)
db.ensureIndex({ fieldName: 'createdAt', expireAfterSeconds: 3600 }, function (err) {
});
// You can also use the option to set an expiration date like so
db.ensureIndex({ fieldName: 'expirationDate', expireAfterSeconds: 0 }, function (err) {
// Now all documents will expire when system time reaches the date in their
// expirationDate field
});
```
**Note:** the `ensureIndex` function creates the index synchronously, so it's best to use it at application startup. It's quite fast so it doesn't increase startup time much (35 ms for a collection containing 10,000 documents).
## Browser version
The browser version and its minified counterpart are in the `browser-version/out` directory. You only need to require `nedb.js` or `nedb.min.js` in your HTML file and the global object `Nedb` can be used right away, with the same API as the server version:
```
<script src="nedb.min.js"></script>
<script>
var db = new Nedb(); // Create an in-memory only datastore
db.insert({ planet: 'Earth' }, function (err) {
db.find({}, function (err, docs) {
// docs contains the two planets Earth and Mars
});
});
</script>
```
If you specify a `filename`, the database will be persistent, and automatically select the best storage method available (IndexedDB, WebSQL or localStorage) depending on the browser. In most cases that means a lot of data can be stored, typically in hundreds of MB. **WARNING**: the storage system changed between v1.3 and v1.4 and is NOT back-compatible! Your application needs to resync client-side when you upgrade NeDB.
NeDB is compatible with all major browsers: Chrome, Safari, Firefox, IE9+. Tests are in the `browser-version/test` directory (files `index.html` and `testPersistence.html`).
If you fork and modify nedb, you can build the browser version from the sources, the build script is `browser-version/build.js`.
## Performance
### Speed
NeDB is not intended to be a replacement of large-scale databases such as MongoDB, and as such was not designed for speed. That said, it is still pretty fast on the expected datasets, especially if you use indexing. On a typical, not-so-fast dev machine, for a collection containing 10,000 documents, with indexing:
* Insert: **10,680 ops/s**
* Find: **43,290 ops/s**
* Update: **8,000 ops/s**
* Remove: **11,750 ops/s**
You can run these simple benchmarks by executing the scripts in the `benchmarks` folder. Run them with the `--help` flag to see how they work.
### Memory footprint
A copy of the whole database is kept in memory. This is not much on the
expected kind of datasets (20MB for 10,000 2KB documents).
## Use in other services
* <a href="https://github.com/louischatriot/connect-nedb-session"
target="_blank">connect-nedb-session</a> is a session store for
Connect and Express, backed by nedb
* If you mostly use NeDB for logging purposes and don't want the memory footprint of your application to grow too large, you can use <a href="https://github.com/louischatriot/nedb-logger" target="_blank">NeDB Logger</a> to insert documents in a NeDB-readable database
* If you've outgrown NeDB, switching to MongoDB won't be too hard as it is the same API. Use <a href="https://github.com/louischatriot/nedb-to-mongodb" target="_blank">this utility</a> to transfer the data from a NeDB database to a MongoDB collection
* An ODM for NeDB: <a href="https://github.com/scottwrobinson/camo" target="_blank">Camo</a>
## Pull requests
**Important: I consider NeDB to be feature-complete, i.e. it does everything I think it should and nothing more. As a general rule I will not accept pull requests anymore, except for bugfixes (of course) or if I get convinced I overlook a strong usecase. Please make sure to open an issue before spending time on any PR.**
If you submit a pull request, thanks! There are a couple rules to follow though to make it manageable:
* The pull request should be atomic, i.e. contain only one feature. If it contains more, please submit multiple pull requests. Reviewing massive, 1000 loc+ pull requests is extremely hard.
* Likewise, if for one unique feature the pull request grows too large (more than 200 loc tests not included), please get in touch first.
* Please stick to the current coding style. It's important that the code uses a coherent style for readability.
* Do not include sylistic improvements ("housekeeping"). If you think one part deserves lots of housekeeping, use a separate pull request so as not to pollute the code.
* Don't forget tests for your new feature. Also don't forget to run the whole test suite before submitting to make sure you didn't introduce regressions.
* Do not build the browser version in your branch, I'll take care of it once the code is merged.
* Update the readme accordingly.
* Last but not least: keep in mind what NeDB's mindset is! The goal is not to be a replacement for MongoDB, but to have a pure JS database, easy to use, cross platform, fast and expressive enough for the target projects (small and self contained apps on server/desktop/browser/mobile). Sometimes it's better to shoot for simplicity than for API completeness with regards to MongoDB.
## Bug reporting guidelines
If you report a bug, thank you! That said for the process to be manageable please strictly adhere to the following guidelines. I'll not be able to handle bug reports that don't:
* Your bug report should be a self-containing gist complete with a package.json for any dependencies you need. I need to run through a simple `git clone gist; npm install; node bugreport.js`, nothing more.
* It should use assertions to showcase the expected vs actual behavior and be hysteresis-proof. It's quite simple in fact, see this example: https://gist.github.com/louischatriot/220cf6bd29c7de06a486
* Simplify as much as you can. Strip all your application-specific code. Most of the time you will see that there is no bug but an error in your code :)
* 50 lines max. If you need more, read the above point and rework your bug report. If you're **really** convinced you need more, please explain precisely in the issue.
* The code should be Javascript, not Coffeescript.
### Bitcoins
You don't have time? You can support NeDB by sending bitcoins to this address: 1dDZLnWpBbodPiN8sizzYrgaz5iahFyb1
## License
See [License](LICENSE)
================================================
FILE: benchmarks/commonUtilities.js
================================================
/**
* Functions that are used in several benchmark tests
*/
var customUtils = require('../lib/customUtils')
, fs = require('fs')
, path = require('path')
, Datastore = require('../lib/datastore')
, Persistence = require('../lib/persistence')
, executeAsap // process.nextTick or setImmediate depending on your Node version
;
try {
executeAsap = setImmediate;
} catch (e) {
executeAsap = process.nextTick;
}
/**
* Configure the benchmark
*/
module.exports.getConfiguration = function (benchDb) {
var d, n
, program = require('commander')
;
program
.option('-n --number [number]', 'Size of the collection to test on', parseInt)
.option('-i --with-index', 'Use an index')
.option('-m --in-memory', 'Test with an in-memory only store')
.parse(process.argv);
n = program.number || 10000;
console.log("----------------------------");
console.log("Test with " + n + " documents");
console.log(program.withIndex ? "Use an index" : "Don't use an index");
console.log(program.inMemory ? "Use an in-memory datastore" : "Use a persistent datastore");
console.log("----------------------------");
d = new Datastore({ filename: benchDb
, inMemoryOnly: program.inMemory
});
return { n: n, d: d, program: program };
};
/**
* Ensure the workspace exists and the db datafile is empty
*/
module.exports.prepareDb = function (filename, cb) {
Persistence.ensureDirectoryExists(path.dirname(filename), function () {
fs.exists(filename, function (exists) {
if (exists) {
fs.unlink(filename, cb);
} else { return cb(); }
});
});
};
/**
* Return an array with the numbers from 0 to n-1, in a random order
* Uses Fisher Yates algorithm
* Useful to get fair tests
*/
function getRandomArray (n) {
var res = []
, i, j, temp
;
for (i = 0; i < n; i += 1) { res[i] = i; }
for (i = n - 1; i >= 1; i -= 1) {
j = Math.floor((i + 1) * Math.random());
temp = res[i];
res[i] = res[j];
res[j] = temp;
}
return res;
};
module.exports.getRandomArray = getRandomArray;
/**
* Insert a certain number of documents for testing
*/
module.exports.insertDocs = function (d, n, profiler, cb) {
var beg = new Date()
, order = getRandomArray(n)
;
profiler.step('Begin inserting ' + n + ' docs');
function runFrom(i) {
if (i === n) { // Finished
var opsPerSecond = Math.floor(1000* n / profiler.elapsedSinceLastStep());
console.log("===== RESULT (insert) ===== " + opsPerSecond + " ops/s");
profiler.step('Finished inserting ' + n + ' docs');
profiler.insertOpsPerSecond = opsPerSecond;
return cb();
}
d.insert({ docNumber: order[i] }, function (err) {
executeAsap(function () {
runFrom(i + 1);
});
});
}
runFrom(0);
};
/**
* Find documents with find
*/
module.exports.findDocs = function (d, n, profiler, cb) {
var beg = new Date()
, order = getRandomArray(n)
;
profiler.step("Finding " + n + " documents");
function runFrom(i) {
if (i === n) { // Finished
console.log("===== RESULT (find) ===== " + Math.floor(1000* n / profiler.elapsedSinceLastStep()) + " ops/s");
profiler.step('Finished finding ' + n + ' docs');
return cb();
}
d.find({ docNumber: order[i] }, function (err, docs) {
if (docs.length !== 1 || docs[0].docNumber !== order[i]) { return cb('One find didnt work'); }
executeAsap(function () {
runFrom(i + 1);
});
});
}
runFrom(0);
};
/**
* Find documents with find and the $in operator
*/
module.exports.findDocsWithIn = function (d, n, profiler, cb) {
var beg = new Date()
, order = getRandomArray(n)
, ins = [], i, j
, arraySize = Math.min(10, n) // The array for $in needs to be smaller than n (inclusive)
;
// Preparing all the $in arrays, will take some time
for (i = 0; i < n; i += 1) {
ins[i] = [];
for (j = 0; j < arraySize; j += 1) {
ins[i].push((i + j) % n);
}
}
profiler.step("Finding " + n + " documents WITH $IN OPERATOR");
function runFrom(i) {
if (i === n) { // Finished
console.log("===== RESULT (find with in selector) ===== " + Math.floor(1000* n / profiler.elapsedSinceLastStep()) + " ops/s");
profiler.step('Finished finding ' + n + ' docs');
return cb();
}
d.find({ docNumber: { $in: ins[i] } }, function (err, docs) {
if (docs.length !== arraySize) { return cb('One find didnt work'); }
executeAsap(function () {
runFrom(i + 1);
});
});
}
runFrom(0);
};
/**
* Find documents with findOne
*/
module.exports.findOneDocs = function (d, n, profiler, cb) {
var beg = new Date()
, order = getRandomArray(n)
;
profiler.step("FindingOne " + n + " documents");
function runFrom(i) {
if (i === n) { // Finished
console.log("===== RESULT (findOne) ===== " + Math.floor(1000* n / profiler.elapsedSinceLastStep()) + " ops/s");
profiler.step('Finished finding ' + n + ' docs');
return cb();
}
d.findOne({ docNumber: order[i] }, function (err, doc) {
if (!doc || doc.docNumber !== order[i]) { return cb('One find didnt work'); }
executeAsap(function () {
runFrom(i + 1);
});
});
}
runFrom(0);
};
/**
* Update documents
* options is the same as the options object for update
*/
module.exports.updateDocs = function (options, d, n, profiler, cb) {
var beg = new Date()
, order = getRandomArray(n)
;
profiler.step("Updating " + n + " documents");
function runFrom(i) {
if (i === n) { // Finished
console.log("===== RESULT (update) ===== " + Math.floor(1000* n / profiler.elapsedSinceLastStep()) + " ops/s");
profiler.step('Finished updating ' + n + ' docs');
return cb();
}
// Will not actually modify the document but will take the same time
d.update({ docNumber: order[i] }, { docNumber: order[i] }, options, function (err, nr) {
if (err) { return cb(err); }
if (nr !== 1) { return cb('One update didnt work'); }
executeAsap(function () {
runFrom(i + 1);
});
});
}
runFrom(0);
};
/**
* Remove documents
* options is the same as the options object for update
*/
module.exports.removeDocs = function (options, d, n, profiler, cb) {
var beg = new Date()
, order = getRandomArray(n)
;
profiler.step("Removing " + n + " documents");
function runFrom(i) {
if (i === n) { // Finished
// opsPerSecond corresponds to 1 insert + 1 remove, needed to keep collection size at 10,000
// We need to subtract the time taken by one insert to get the time actually taken by one remove
var opsPerSecond = Math.floor(1000 * n / profiler.elapsedSinceLastStep());
var removeOpsPerSecond = Math.floor(1 / ((1 / opsPerSecond) - (1 / profiler.insertOpsPerSecond)))
console.log("===== RESULT (remove) ===== " + removeOpsPerSecond + " ops/s");
profiler.step('Finished removing ' + n + ' docs');
return cb();
}
d.remove({ docNumber: order[i] }, options, function (err, nr) {
if (err) { return cb(err); }
if (nr !== 1) { return cb('One remove didnt work'); }
d.insert({ docNumber: order[i] }, function (err) { // We need to reinsert the doc so that we keep the collection's size at n
// So actually we're calculating the average time taken by one insert + one remove
executeAsap(function () {
runFrom(i + 1);
});
});
});
}
runFrom(0);
};
/**
* Load database
*/
module.exports.loadDatabase = function (d, n, profiler, cb) {
var beg = new Date()
, order = getRandomArray(n)
;
profiler.step("Loading the database " + n + " times");
function runFrom(i) {
if (i === n) { // Finished
console.log("===== RESULT ===== " + Math.floor(1000* n / profiler.elapsedSinceLastStep()) + " ops/s");
profiler.step('Finished loading a database' + n + ' times');
return cb();
}
d.loadDatabase(function (err) {
executeAsap(function () {
runFrom(i + 1);
});
});
}
runFrom(0);
};
================================================
FILE: benchmarks/ensureIndex.js
================================================
var Datastore = require('../lib/datastore')
, benchDb = 'workspace/insert.bench.db'
, async = require('async')
, commonUtilities = require('./commonUtilities')
, execTime = require('exec-time')
, profiler = new execTime('INSERT BENCH')
, d = new Datastore(benchDb)
, program = require('commander')
, n
;
program
.option('-n --number [number]', 'Size of the collection to test on', parseInt)
.option('-i --with-index', 'Test with an index')
.parse(process.argv);
n = program.number || 10000;
console.log("----------------------------");
console.log("Test with " + n + " documents");
console.log("----------------------------");
async.waterfall([
async.apply(commonUtilities.prepareDb, benchDb)
, function (cb) {
d.loadDatabase(function (err) {
if (err) { return cb(err); }
cb();
});
}
, function (cb) { profiler.beginProfiling(); return cb(); }
, async.apply(commonUtilities.insertDocs, d, n, profiler)
, function (cb) {
var i;
profiler.step('Begin calling ensureIndex ' + n + ' times');
for (i = 0; i < n; i += 1) {
d.ensureIndex({ fieldName: 'docNumber' });
delete d.indexes.docNumber;
}
console.log("Average time for one ensureIndex: " + (profiler.elapsedSinceLastStep() / n) + "ms");
profiler.step('Finished calling ensureIndex ' + n + ' times');
}
], function (err) {
profiler.step("Benchmark finished");
if (err) { return console.log("An error was encountered: ", err); }
});
================================================
FILE: benchmarks/find.js
================================================
var Datastore = require('../lib/datastore')
, benchDb = 'workspace/find.bench.db'
, fs = require('fs')
, path = require('path')
, async = require('async')
, execTime = require('exec-time')
, profiler = new execTime('FIND BENCH')
, commonUtilities = require('./commonUtilities')
, config = commonUtilities.getConfiguration(benchDb)
, d = config.d
, n = config.n
;
async.waterfall([
async.apply(commonUtilities.prepareDb, benchDb)
, function (cb) {
d.loadDatabase(function (err) {
if (err) { return cb(err); }
if (config.program.withIndex) { d.ensureIndex({ fieldName: 'docNumber' }); }
cb();
});
}
, function (cb) { profiler.beginProfiling(); return cb(); }
, async.apply(commonUtilities.insertDocs, d, n, profiler)
, async.apply(commonUtilities.findDocs, d, n, profiler)
], function (err) {
profiler.step("Benchmark finished");
if (err) { return console.log("An error was encountered: ", err); }
});
================================================
FILE: benchmarks/findOne.js
================================================
var Datastore = require('../lib/datastore')
, benchDb = 'workspace/findOne.bench.db'
, fs = require('fs')
, path = require('path')
, async = require('async')
, execTime = require('exec-time')
, profiler = new execTime('FINDONE BENCH')
, commonUtilities = require('./commonUtilities')
, config = commonUtilities.getConfiguration(benchDb)
, d = config.d
, n = config.n
;
async.waterfall([
async.apply(commonUtilities.prepareDb, benchDb)
, function (cb) {
d.loadDatabase(function (err) {
if (err) { return cb(err); }
if (config.program.withIndex) { d.ensureIndex({ fieldName: 'docNumber' }); }
cb();
});
}
, function (cb) { profiler.beginProfiling(); return cb(); }
, async.apply(commonUtilities.insertDocs, d, n, profiler)
, function (cb) { setTimeout(function () {cb();}, 500); }
, async.apply(commonUtilities.findOneDocs, d, n, profiler)
], function (err) {
profiler.step("Benchmark finished");
if (err) { return console.log("An error was encountered: ", err); }
});
================================================
FILE: benchmarks/findWithIn.js
================================================
var Datastore = require('../lib/datastore')
, benchDb = 'workspace/find.bench.db'
, fs = require('fs')
, path = require('path')
, async = require('async')
, execTime = require('exec-time')
, profiler = new execTime('FIND BENCH')
, commonUtilities = require('./commonUtilities')
, config = commonUtilities.getConfiguration(benchDb)
, d = config.d
, n = config.n
;
async.waterfall([
async.apply(commonUtilities.prepareDb, benchDb)
, function (cb) {
d.loadDatabase(function (err) {
if (err) { return cb(err); }
if (config.program.withIndex) { d.ensureIndex({ fieldName: 'docNumber' }); }
cb();
});
}
, function (cb) { profiler.beginProfiling(); return cb(); }
, async.apply(commonUtilities.insertDocs, d, n, profiler)
, async.apply(commonUtilities.findDocsWithIn, d, n, profiler)
], function (err) {
profiler.step("Benchmark finished");
if (err) { return console.log("An error was encountered: ", err); }
});
================================================
FILE: benchmarks/insert.js
================================================
var Datastore = require('../lib/datastore')
, benchDb = 'workspace/insert.bench.db'
, async = require('async')
, execTime = require('exec-time')
, profiler = new execTime('INSERT BENCH')
, commonUtilities = require('./commonUtilities')
, config = commonUtilities.getConfiguration(benchDb)
, d = config.d
, n = config.n
;
async.waterfall([
async.apply(commonUtilities.prepareDb, benchDb)
, function (cb) {
d.loadDatabase(function (err) {
if (err) { return cb(err); }
if (config.program.withIndex) {
d.ensureIndex({ fieldName: 'docNumber' });
n = 2 * n; // We will actually insert twice as many documents
// because the index is slower when the collection is already
// big. So the result given by the algorithm will be a bit worse than
// actual performance
}
cb();
});
}
, function (cb) { profiler.beginProfiling(); return cb(); }
, async.apply(commonUtilities.insertDocs, d, n, profiler)
], function (err) {
profiler.step("Benchmark finished");
if (err) { return console.log("An error was encountered: ", err); }
});
================================================
FILE: benchmarks/loadDatabase.js
================================================
var Datastore = require('../lib/datastore')
, benchDb = 'workspace/loaddb.bench.db'
, fs = require('fs')
, path = require('path')
, async = require('async')
, commonUtilities = require('./commonUtilities')
, execTime = require('exec-time')
, profiler = new execTime('LOADDB BENCH')
, d = new Datastore(benchDb)
, program = require('commander')
, n
;
program
.option('-n --number [number]', 'Size of the collection to test on', parseInt)
.option('-i --with-index', 'Test with an index')
.parse(process.argv);
n = program.number || 10000;
console.log("----------------------------");
console.log("Test with " + n + " documents");
console.log(program.withIndex ? "Use an index" : "Don't use an index");
console.log("----------------------------");
async.waterfall([
async.apply(commonUtilities.prepareDb, benchDb)
, function (cb) {
d.loadDatabase(cb);
}
, function (cb) { profiler.beginProfiling(); return cb(); }
, async.apply(commonUtilities.insertDocs, d, n, profiler)
, async.apply(commonUtilities.loadDatabase, d, n, profiler)
], function (err) {
profiler.step("Benchmark finished");
if (err) { return console.log("An error was encountered: ", err); }
});
================================================
FILE: benchmarks/remove.js
================================================
var Datastore = require('../lib/datastore')
, benchDb = 'workspace/remove.bench.db'
, fs = require('fs')
, path = require('path')
, async = require('async')
, execTime = require('exec-time')
, profiler = new execTime('REMOVE BENCH')
, commonUtilities = require('./commonUtilities')
, config = commonUtilities.getConfiguration(benchDb)
, d = config.d
, n = config.n
;
async.waterfall([
async.apply(commonUtilities.prepareDb, benchDb)
, function (cb) {
d.loadDatabase(function (err) {
if (err) { return cb(err); }
if (config.program.withIndex) { d.ensureIndex({ fieldName: 'docNumber' }); }
cb();
});
}
, function (cb) { profiler.beginProfiling(); return cb(); }
, async.apply(commonUtilities.insertDocs, d, n, profiler)
// Test with remove only one document
, function (cb) { profiler.step('MULTI: FALSE'); return cb(); }
, async.apply(commonUtilities.removeDocs, { multi: false }, d, n, profiler)
// Test with multiple documents
, function (cb) { d.remove({}, { multi: true }, function () { return cb(); }); }
, async.apply(commonUtilities.insertDocs, d, n, profiler)
, function (cb) { profiler.step('MULTI: TRUE'); return cb(); }
, async.apply(commonUtilities.removeDocs, { multi: true }, d, n, profiler)
], function (err) {
profiler.step("Benchmark finished");
if (err) { return console.log("An error was encountered: ", err); }
});
================================================
FILE: benchmarks/update.js
================================================
var Datastore = require('../lib/datastore')
, benchDb = 'workspace/update.bench.db'
, fs = require('fs')
, path = require('path')
, async = require('async')
, execTime = require('exec-time')
, profiler = new execTime('UPDATE BENCH')
, commonUtilities = require('./commonUtilities')
, config = commonUtilities.getConfiguration(benchDb)
, d = config.d
, n = config.n
;
async.waterfall([
async.apply(commonUtilities.prepareDb, benchDb)
, function (cb) {
d.loadDatabase(function (err) {
if (err) { return cb(err); }
if (config.program.withIndex) { d.ensureIndex({ fieldName: 'docNumber' }); }
cb();
});
}
, function (cb) { profiler.beginProfiling(); return cb(); }
, async.apply(commonUtilities.insertDocs, d, n, profiler)
// Test with update only one document
, function (cb) { profiler.step('MULTI: FALSE'); return cb(); }
, async.apply(commonUtilities.updateDocs, { multi: false }, d, n, profiler)
// Test with multiple documents
, function (cb) { d.remove({}, { multi: true }, function (err) { return cb(); }); }
, async.apply(commonUtilities.insertDocs, d, n, profiler)
, function (cb) { profiler.step('MULTI: TRUE'); return cb(); }
, async.apply(commonUtilities.updateDocs, { multi: true }, d, n, profiler)
], function (err) {
profiler.step("Benchmark finished");
if (err) { return console.log("An error was encountered: ", err); }
});
================================================
FILE: bower.json
================================================
{
"name": "nedb",
"description": "The Javascript Database for Node, nwjs, Electron and the browser",
"ignore": ["benchmarks", "lib", "test", "test_lac"],
"main": ["browser-version/nedb.js", "browser-version/nedb.min.js"]
}
================================================
FILE: browser-version/browser-specific/lib/customUtils.js
================================================
/**
* Specific customUtils for the browser, where we don't have access to the Crypto and Buffer modules
*/
/**
* Taken from the crypto-browserify module
* https://github.com/dominictarr/crypto-browserify
* NOTE: Math.random() does not guarantee "cryptographic quality" but we actually don't need it
*/
function randomBytes (size) {
var bytes = new Array(size);
var r;
for (var i = 0, r; i < size; i++) {
if ((i & 0x03) == 0) r = Math.random() * 0x100000000;
bytes[i] = r >>> ((i & 0x03) << 3) & 0xff;
}
return bytes;
}
/**
* Taken from the base64-js module
* https://github.com/beatgammit/base64-js/
*/
function byteArrayToBase64 (uint8) {
var lookup = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/'
, extraBytes = uint8.length % 3 // if we have 1 byte left, pad 2 bytes
, output = ""
, temp, length, i;
function tripletToBase64 (num) {
return lookup[num >> 18 & 0x3F] + lookup[num >> 12 & 0x3F] + lookup[num >> 6 & 0x3F] + lookup[num & 0x3F];
};
// go through the array every three bytes, we'll deal with trailing stuff later
for (i = 0, length = uint8.length - extraBytes; i < length; i += 3) {
temp = (uint8[i] << 16) + (uint8[i + 1] << 8) + (uint8[i + 2]);
output += tripletToBase64(temp);
}
// pad the end with zeros, but make sure to not forget the extra bytes
switch (extraBytes) {
case 1:
temp = uint8[uint8.length - 1];
output += lookup[temp >> 2];
output += lookup[(temp << 4) & 0x3F];
output += '==';
break;
case 2:
temp = (uint8[uint8.length - 2] << 8) + (uint8[uint8.length - 1]);
output += lookup[temp >> 10];
output += lookup[(temp >> 4) & 0x3F];
output += lookup[(temp << 2) & 0x3F];
output += '=';
break;
}
return output;
}
/**
* Return a random alphanumerical string of length len
* There is a very small probability (less than 1/1,000,000) for the length to be less than len
* (il the base64 conversion yields too many pluses and slashes) but
* that's not an issue here
* The probability of a collision is extremely small (need 3*10^12 documents to have one chance in a million of a collision)
* See http://en.wikipedia.org/wiki/Birthday_problem
*/
function uid (len) {
return byteArrayToBase64(randomBytes(Math.ceil(Math.max(8, len * 2)))).replace(/[+\/]/g, '').slice(0, len);
}
module.exports.uid = uid;
================================================
FILE: browser-version/browser-specific/lib/storage.js
================================================
/**
* Way data is stored for this database
* For a Node.js/Node Webkit database it's the file system
* For a browser-side database it's localforage, which uses the best backend available (IndexedDB then WebSQL then localStorage)
*
* This version is the browser version
*/
var localforage = require('localforage')
// Configure localforage to display NeDB name for now. Would be a good idea to let user use his own app name
localforage.config({
name: 'NeDB'
, storeName: 'nedbdata'
});
function exists (filename, callback) {
localforage.getItem(filename, function (err, value) {
if (value !== null) { // Even if value is undefined, localforage returns null
return callback(true);
} else {
return callback(false);
}
});
}
function rename (filename, newFilename, callback) {
localforage.getItem(filename, function (err, value) {
if (value === null) {
localforage.removeItem(newFilename, function () { return callback(); });
} else {
localforage.setItem(newFilename, value, function () {
localforage.removeItem(filename, function () { return callback(); });
});
}
});
}
function writeFile (filename, contents, options, callback) {
// Options do not matter in browser setup
if (typeof options === 'function') { callback = options; }
localforage.setItem(filename, contents, function () { return callback(); });
}
function appendFile (filename, toAppend, options, callback) {
// Options do not matter in browser setup
if (typeof options === 'function') { callback = options; }
localforage.getItem(filename, function (err, contents) {
contents = contents || '';
contents += toAppend;
localforage.setItem(filename, contents, function () { return callback(); });
});
}
function readFile (filename, options, callback) {
// Options do not matter in browser setup
if (typeof options === 'function') { callback = options; }
localforage.getItem(filename, function (err, contents) { return callback(null, contents || ''); });
}
function unlink (filename, callback) {
localforage.removeItem(filename, function () { return callback(); });
}
// Nothing to do, no directories will be used on the browser
function mkdirp (dir, callback) {
return callback();
}
// Nothing to do, no data corruption possible in the brower
function ensureDatafileIntegrity (filename, callback) {
return callback(null);
}
// Interface
module.exports.exists = exists;
module.exports.rename = rename;
module.exports.writeFile = writeFile;
module.exports.crashSafeWriteFile = writeFile; // No need for a crash safe function in the browser
module.exports.appendFile = appendFile;
module.exports.readFile = readFile;
module.exports.unlink = unlink;
module.exports.mkdirp = mkdirp;
module.exports.ensureDatafileIntegrity = ensureDatafileIntegrity;
================================================
FILE: browser-version/build.js
================================================
/**
* Build the browser version of nedb
*/
var fs = require('fs')
, path = require('path')
, child_process = require('child_process')
, toCopy = ['lib', 'node_modules']
, async, browserify, uglify
;
// Ensuring both node_modules (the source one and build one), src and out directories exist
function ensureDirExists (name) {
try {
fs.mkdirSync(path.join(__dirname, name));
} catch (e) {
if (e.code !== 'EEXIST') {
console.log("Error ensuring that node_modules exists");
process.exit(1);
}
}
}
ensureDirExists('../node_modules');
ensureDirExists('node_modules');
ensureDirExists('out');
ensureDirExists('src');
// Installing build dependencies and require them
console.log("Installing build dependencies");
child_process.exec('npm install', { cwd: __dirname }, function (err, stdout, stderr) {
if (err) { console.log("Error reinstalling dependencies"); process.exit(1); }
fs = require('fs-extra');
async = require('async');
browserify = require('browserify');
uglify = require('uglify-js');
async.waterfall([
function (cb) {
console.log("Installing source dependencies if needed");
child_process.exec('npm install', { cwd: path.join(__dirname, '..') }, function (err) { return cb(err); });
}
, function (cb) {
console.log("Removing contents of the src directory");
async.eachSeries(fs.readdirSync(path.join(__dirname, 'src')), function (item, _cb) {
fs.remove(path.join(__dirname, 'src', item), _cb);
}, cb);
}
, function (cb) {
console.log("Copying source files");
async.eachSeries(toCopy, function (item, _cb) {
fs.copy(path.join(__dirname, '..', item), path.join(__dirname, 'src', item), _cb);
}, cb);
}
, function (cb) {
console.log("Copying browser specific files to replace their server-specific counterparts");
async.eachSeries(fs.readdirSync(path.join(__dirname, 'browser-specific')), function (item, _cb) {
fs.copy(path.join(__dirname, 'browser-specific', item), path.join(__dirname, 'src', item), _cb);
}, cb);
}
, function (cb) {
console.log("Browserifying the code");
var b = browserify()
, srcPath = path.join(__dirname, 'src/lib/datastore.js');
b.add(srcPath);
b.bundle({ standalone: 'Nedb' }, function (err, out) {
if (err) { return cb(err); }
fs.writeFile(path.join(__dirname, 'out/nedb.js'), out, 'utf8', function (err) {
if (err) {
return cb(err);
} else {
return cb(null, out);
}
});
});
}
, function (out, cb) {
console.log("Creating the minified version");
var compressedCode = uglify.minify(out, { fromString: true });
fs.writeFile(path.join(__dirname, 'out/nedb.min.js'), compressedCode.code, 'utf8', cb);
}
], function (err) {
if (err) {
console.log("Error during build");
console.log(err);
} else {
console.log("Build finished with success");
}
});
});
================================================
FILE: browser-version/out/nedb.js
================================================
(function(e){if("function"==typeof bootstrap)bootstrap("nedb",e);else if("object"==typeof exports)module.exports=e();else if("function"==typeof define&&define.amd)define(e);else if("undefined"!=typeof ses){if(!ses.ok())return;ses.makeNedb=e}else"undefined"!=typeof window?window.Nedb=e():global.Nedb=e()})(function(){var define,ses,bootstrap,module,exports;
return (function(e,t,n){function i(n,s){if(!t[n]){if(!e[n]){var o=typeof require=="function"&&require;if(!s&&o)return o(n,!0);if(r)return r(n,!0);throw new Error("Cannot find module '"+n+"'")}var u=t[n]={exports:{}};e[n][0].call(u.exports,function(t){var r=e[n][1][t];return i(r?r:t)},u,u.exports)}return t[n].exports}var r=typeof require=="function"&&require;for(var s=0;s<n.length;s++)i(n[s]);return i})({1:[function(require,module,exports){
var process=require("__browserify_process");if (!process.EventEmitter) process.EventEmitter = function () {};
var EventEmitter = exports.EventEmitter = process.EventEmitter;
var isArray = typeof Array.isArray === 'function'
? Array.isArray
: function (xs) {
return Object.prototype.toString.call(xs) === '[object Array]'
}
;
function indexOf (xs, x) {
if (xs.indexOf) return xs.indexOf(x);
for (var i = 0; i < xs.length; i++) {
if (x === xs[i]) return i;
}
return -1;
}
// By default EventEmitters will print a warning if more than
// 10 listeners are added to it. This is a useful default which
// helps finding memory leaks.
//
// Obviously not all Emitters should be limited to 10. This function allows
// that to be increased. Set to zero for unlimited.
var defaultMaxListeners = 10;
EventEmitter.prototype.setMaxListeners = function(n) {
if (!this._events) this._events = {};
this._events.maxListeners = n;
};
EventEmitter.prototype.emit = function(type) {
// If there is no 'error' event listener then throw.
if (type === 'error') {
if (!this._events || !this._events.error ||
(isArray(this._events.error) && !this._events.error.length))
{
if (arguments[1] instanceof Error) {
throw arguments[1]; // Unhandled 'error' event
} else {
throw new Error("Uncaught, unspecified 'error' event.");
}
return false;
}
}
if (!this._events) return false;
var handler = this._events[type];
if (!handler) return false;
if (typeof handler == 'function') {
switch (arguments.length) {
// fast cases
case 1:
handler.call(this);
break;
case 2:
handler.call(this, arguments[1]);
break;
case 3:
handler.call(this, arguments[1], arguments[2]);
break;
// slower
default:
var args = Array.prototype.slice.call(arguments, 1);
handler.apply(this, args);
}
return true;
} else if (isArray(handler)) {
var args = Array.prototype.slice.call(arguments, 1);
var listeners = handler.slice();
for (var i = 0, l = listeners.length; i < l; i++) {
listeners[i].apply(this, args);
}
return true;
} else {
return false;
}
};
// EventEmitter is defined in src/node_events.cc
// EventEmitter.prototype.emit() is also defined there.
EventEmitter.prototype.addListener = function(type, listener) {
if ('function' !== typeof listener) {
throw new Error('addListener only takes instances of Function');
}
if (!this._events) this._events = {};
// To avoid recursion in the case that type == "newListeners"! Before
// adding it to the listeners, first emit "newListeners".
this.emit('newListener', type, listener);
if (!this._events[type]) {
// Optimize the case of one listener. Don't need the extra array object.
this._events[type] = listener;
} else if (isArray(this._events[type])) {
// Check for listener leak
if (!this._events[type].warned) {
var m;
if (this._events.maxListeners !== undefined) {
m = this._events.maxListeners;
} else {
m = defaultMaxListeners;
}
if (m && m > 0 && this._events[type].length > m) {
this._events[type].warned = true;
console.error('(node) warning: possible EventEmitter memory ' +
'leak detected. %d listeners added. ' +
'Use emitter.setMaxListeners() to increase limit.',
this._events[type].length);
console.trace();
}
}
// If we've already got an array, just append.
this._events[type].push(listener);
} else {
// Adding the second element, need to change to array.
this._events[type] = [this._events[type], listener];
}
return this;
};
EventEmitter.prototype.on = EventEmitter.prototype.addListener;
EventEmitter.prototype.once = function(type, listener) {
var self = this;
self.on(type, function g() {
self.removeListener(type, g);
listener.apply(this, arguments);
});
return this;
};
EventEmitter.prototype.removeListener = function(type, listener) {
if ('function' !== typeof listener) {
throw new Error('removeListener only takes instances of Function');
}
// does not use listeners(), so no side effect of creating _events[type]
if (!this._events || !this._events[type]) return this;
var list = this._events[type];
if (isArray(list)) {
var i = indexOf(list, listener);
if (i < 0) return this;
list.splice(i, 1);
if (list.length == 0)
delete this._events[type];
} else if (this._events[type] === listener) {
delete this._events[type];
}
return this;
};
EventEmitter.prototype.removeAllListeners = function(type) {
if (arguments.length === 0) {
this._events = {};
return this;
}
// does not use listeners(), so no side effect of creating _events[type]
if (type && this._events && this._events[type]) this._events[type] = null;
return this;
};
EventEmitter.prototype.listeners = function(type) {
if (!this._events) this._events = {};
if (!this._events[type]) this._events[type] = [];
if (!isArray(this._events[type])) {
this._events[type] = [this._events[type]];
}
return this._events[type];
};
EventEmitter.listenerCount = function(emitter, type) {
var ret;
if (!emitter._events || !emitter._events[type])
ret = 0;
else if (typeof emitter._events[type] === 'function')
ret = 1;
else
ret = emitter._events[type].length;
return ret;
};
},{"__browserify_process":4}],2:[function(require,module,exports){
var process=require("__browserify_process");function filter (xs, fn) {
var res = [];
for (var i = 0; i < xs.length; i++) {
if (fn(xs[i], i, xs)) res.push(xs[i]);
}
return res;
}
// resolves . and .. elements in a path array with directory names there
// must be no slashes, empty elements, or device names (c:\) in the array
// (so also no leading and trailing slashes - it does not distinguish
// relative and absolute paths)
function normalizeArray(parts, allowAboveRoot) {
// if the path tries to go above the root, `up` ends up > 0
var up = 0;
for (var i = parts.length; i >= 0; i--) {
var last = parts[i];
if (last == '.') {
parts.splice(i, 1);
} else if (last === '..') {
parts.splice(i, 1);
up++;
} else if (up) {
parts.splice(i, 1);
up--;
}
}
// if the path is allowed to go above the root, restore leading ..s
if (allowAboveRoot) {
for (; up--; up) {
parts.unshift('..');
}
}
return parts;
}
// Regex to split a filename into [*, dir, basename, ext]
// posix version
var splitPathRe = /^(.+\/(?!$)|\/)?((?:.+?)?(\.[^.]*)?)$/;
// path.resolve([from ...], to)
// posix version
exports.resolve = function() {
var resolvedPath = '',
resolvedAbsolute = false;
for (var i = arguments.length; i >= -1 && !resolvedAbsolute; i--) {
var path = (i >= 0)
? arguments[i]
: process.cwd();
// Skip empty and invalid entries
if (typeof path !== 'string' || !path) {
continue;
}
resolvedPath = path + '/' + resolvedPath;
resolvedAbsolute = path.charAt(0) === '/';
}
// At this point the path should be resolved to a full absolute path, but
// handle relative paths to be safe (might happen when process.cwd() fails)
// Normalize the path
resolvedPath = normalizeArray(filter(resolvedPath.split('/'), function(p) {
return !!p;
}), !resolvedAbsolute).join('/');
return ((resolvedAbsolute ? '/' : '') + resolvedPath) || '.';
};
// path.normalize(path)
// posix version
exports.normalize = function(path) {
var isAbsolute = path.charAt(0) === '/',
trailingSlash = path.slice(-1) === '/';
// Normalize the path
path = normalizeArray(filter(path.split('/'), function(p) {
return !!p;
}), !isAbsolute).join('/');
if (!path && !isAbsolute) {
path = '.';
}
if (path && trailingSlash) {
path += '/';
}
return (isAbsolute ? '/' : '') + path;
};
// posix version
exports.join = function() {
var paths = Array.prototype.slice.call(arguments, 0);
return exports.normalize(filter(paths, function(p, index) {
return p && typeof p === 'string';
}).join('/'));
};
exports.dirname = function(path) {
var dir = splitPathRe.exec(path)[1] || '';
var isWindows = false;
if (!dir) {
// No dirname
return '.';
} else if (dir.length === 1 ||
(isWindows && dir.length <= 3 && dir.charAt(1) === ':')) {
// It is just a slash or a drive letter with a slash
return dir;
} else {
// It is a full dirname, strip trailing slash
return dir.substring(0, dir.length - 1);
}
};
exports.basename = function(path, ext) {
var f = splitPathRe.exec(path)[2] || '';
// TODO: make this comparison case-insensitive on windows?
if (ext && f.substr(-1 * ext.length) === ext) {
f = f.substr(0, f.length - ext.length);
}
return f;
};
exports.extname = function(path) {
return splitPathRe.exec(path)[3] || '';
};
exports.relative = function(from, to) {
from = exports.resolve(from).substr(1);
to = exports.resolve(to).substr(1);
function trim(arr) {
var start = 0;
for (; start < arr.length; start++) {
if (arr[start] !== '') break;
}
var end = arr.length - 1;
for (; end >= 0; end--) {
if (arr[end] !== '') break;
}
if (start > end) return [];
return arr.slice(start, end - start + 1);
}
var fromParts = trim(from.split('/'));
var toParts = trim(to.split('/'));
var length = Math.min(fromParts.length, toParts.length);
var samePartsLength = length;
for (var i = 0; i < length; i++) {
if (fromParts[i] !== toParts[i]) {
samePartsLength = i;
break;
}
}
var outputParts = [];
for (var i = samePartsLength; i < fromParts.length; i++) {
outputParts.push('..');
}
outputParts = outputParts.concat(toParts.slice(samePartsLength));
return outputParts.join('/');
};
exports.sep = '/';
},{"__browserify_process":4}],3:[function(require,module,exports){
var events = require('events');
exports.isArray = isArray;
exports.isDate = function(obj){return Object.prototype.toString.call(obj) === '[object Date]'};
exports.isRegExp = function(obj){return Object.prototype.toString.call(obj) === '[object RegExp]'};
exports.print = function () {};
exports.puts = function () {};
exports.debug = function() {};
exports.inspect = function(obj, showHidden, depth, colors) {
var seen = [];
var stylize = function(str, styleType) {
// http://en.wikipedia.org/wiki/ANSI_escape_code#graphics
var styles =
{ 'bold' : [1, 22],
'italic' : [3, 23],
'underline' : [4, 24],
'inverse' : [7, 27],
'white' : [37, 39],
'grey' : [90, 39],
'black' : [30, 39],
'blue' : [34, 39],
'cyan' : [36, 39],
'green' : [32, 39],
'magenta' : [35, 39],
'red' : [31, 39],
'yellow' : [33, 39] };
var style =
{ 'special': 'cyan',
'number': 'blue',
'boolean': 'yellow',
'undefined': 'grey',
'null': 'bold',
'string': 'green',
'date': 'magenta',
// "name": intentionally not styling
'regexp': 'red' }[styleType];
if (style) {
return '\u001b[' + styles[style][0] + 'm' + str +
'\u001b[' + styles[style][1] + 'm';
} else {
return str;
}
};
if (! colors) {
stylize = function(str, styleType) { return str; };
}
function format(value, recurseTimes) {
// Provide a hook for user-specified inspect functions.
// Check that value is an object with an inspect function on it
if (value && typeof value.inspect === 'function' &&
// Filter out the util module, it's inspect function is special
value !== exports &&
// Also filter out any prototype objects using the circular check.
!(value.constructor && value.constructor.prototype === value)) {
return value.inspect(recurseTimes);
}
// Primitive types cannot have properties
switch (typeof value) {
case 'undefined':
return stylize('undefined', 'undefined');
case 'string':
var simple = '\'' + JSON.stringify(value).replace(/^"|"$/g, '')
.replace(/'/g, "\\'")
.replace(/\\"/g, '"') + '\'';
return stylize(simple, 'string');
case 'number':
return stylize('' + value, 'number');
case 'boolean':
return stylize('' + value, 'boolean');
}
// For some reason typeof null is "object", so special case here.
if (value === null) {
return stylize('null', 'null');
}
// Look up the keys of the object.
var visible_keys = Object_keys(value);
var keys = showHidden ? Object_getOwnPropertyNames(value) : visible_keys;
// Functions without properties can be shortcutted.
if (typeof value === 'function' && keys.length === 0) {
if (isRegExp(value)) {
return stylize('' + value, 'regexp');
} else {
var name = value.name ? ': ' + value.name : '';
return stylize('[Function' + name + ']', 'special');
}
}
// Dates without properties can be shortcutted
if (isDate(value) && keys.length === 0) {
return stylize(value.toUTCString(), 'date');
}
var base, type, braces;
// Determine the object type
if (isArray(value)) {
type = 'Array';
braces = ['[', ']'];
} else {
type = 'Object';
braces = ['{', '}'];
}
// Make functions say that they are functions
if (typeof value === 'function') {
var n = value.name ? ': ' + value.name : '';
base = (isRegExp(value)) ? ' ' + value : ' [Function' + n + ']';
} else {
base = '';
}
// Make dates with properties first say the date
if (isDate(value)) {
base = ' ' + value.toUTCString();
}
if (keys.length === 0) {
return braces[0] + base + braces[1];
}
if (recurseTimes < 0) {
if (isRegExp(value)) {
return stylize('' + value, 'regexp');
} else {
return stylize('[Object]', 'special');
}
}
seen.push(value);
var output = keys.map(function(key) {
var name, str;
if (value.__lookupGetter__) {
if (value.__lookupGetter__(key)) {
if (value.__lookupSetter__(key)) {
str = stylize('[Getter/Setter]', 'special');
} else {
str = stylize('[Getter]', 'special');
}
} else {
if (value.__lookupSetter__(key)) {
str = stylize('[Setter]', 'special');
}
}
}
if (visible_keys.indexOf(key) < 0) {
name = '[' + key + ']';
}
if (!str) {
if (seen.indexOf(value[key]) < 0) {
if (recurseTimes === null) {
str = format(value[key]);
} else {
str = format(value[key], recurseTimes - 1);
}
if (str.indexOf('\n') > -1) {
if (isArray(value)) {
str = str.split('\n').map(function(line) {
return ' ' + line;
}).join('\n').substr(2);
} else {
str = '\n' + str.split('\n').map(function(line) {
return ' ' + line;
}).join('\n');
}
}
} else {
str = stylize('[Circular]', 'special');
}
}
if (typeof name === 'undefined') {
if (type === 'Array' && key.match(/^\d+$/)) {
return str;
}
name = JSON.stringify('' + key);
if (name.match(/^"([a-zA-Z_][a-zA-Z_0-9]*)"$/)) {
name = name.substr(1, name.length - 2);
name = stylize(name, 'name');
} else {
name = name.replace(/'/g, "\\'")
.replace(/\\"/g, '"')
.replace(/(^"|"$)/g, "'");
name = stylize(name, 'string');
}
}
return name + ': ' + str;
});
seen.pop();
var numLinesEst = 0;
var length = output.reduce(function(prev, cur) {
numLinesEst++;
if (cur.indexOf('\n') >= 0) numLinesEst++;
return prev + cur.length + 1;
}, 0);
if (length > 50) {
output = braces[0] +
(base === '' ? '' : base + '\n ') +
' ' +
output.join(',\n ') +
' ' +
braces[1];
} else {
output = braces[0] + base + ' ' + output.join(', ') + ' ' + braces[1];
}
return output;
}
return format(obj, (typeof depth === 'undefined' ? 2 : depth));
};
function isArray(ar) {
return Array.isArray(ar) ||
(typeof ar === 'object' && Object.prototype.toString.call(ar) === '[object Array]');
}
function isRegExp(re) {
typeof re === 'object' && Object.prototype.toString.call(re) === '[object RegExp]';
}
function isDate(d) {
return typeof d === 'object' && Object.prototype.toString.call(d) === '[object Date]';
}
function pad(n) {
return n < 10 ? '0' + n.toString(10) : n.toString(10);
}
var months = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep',
'Oct', 'Nov', 'Dec'];
// 26 Feb 16:19:34
function timestamp() {
var d = new Date();
var time = [pad(d.getHours()),
pad(d.getMinutes()),
pad(d.getSeconds())].join(':');
return [d.getDate(), months[d.getMonth()], time].join(' ');
}
exports.log = function (msg) {};
exports.pump = null;
var Object_keys = Object.keys || function (obj) {
var res = [];
for (var key in obj) res.push(key);
return res;
};
var Object_getOwnPropertyNames = Object.getOwnPropertyNames || function (obj) {
var res = [];
for (var key in obj) {
if (Object.hasOwnProperty.call(obj, key)) res.push(key);
}
return res;
};
var Object_create = Object.create || function (prototype, properties) {
// from es5-shim
var object;
if (prototype === null) {
object = { '__proto__' : null };
}
else {
if (typeof prototype !== 'object') {
throw new TypeError(
'typeof prototype[' + (typeof prototype) + '] != \'object\''
);
}
var Type = function () {};
Type.prototype = prototype;
object = new Type();
object.__proto__ = prototype;
}
if (typeof properties !== 'undefined' && Object.defineProperties) {
Object.defineProperties(object, properties);
}
return object;
};
exports.inherits = function(ctor, superCtor) {
ctor.super_ = superCtor;
ctor.prototype = Object_create(superCtor.prototype, {
constructor: {
value: ctor,
enumerable: false,
writable: true,
configurable: true
}
});
};
var formatRegExp = /%[sdj%]/g;
exports.format = function(f) {
if (typeof f !== 'string') {
var objects = [];
for (var i = 0; i < arguments.length; i++) {
objects.push(exports.inspect(arguments[i]));
}
return objects.join(' ');
}
var i = 1;
var args = arguments;
var len = args.length;
var str = String(f).replace(formatRegExp, function(x) {
if (x === '%%') return '%';
if (i >= len) return x;
switch (x) {
case '%s': return String(args[i++]);
case '%d': return Number(args[i++]);
case '%j': return JSON.stringify(args[i++]);
default:
return x;
}
});
for(var x = args[i]; i < len; x = args[++i]){
if (x === null || typeof x !== 'object') {
str += ' ' + x;
} else {
str += ' ' + exports.inspect(x);
}
}
return str;
};
},{"events":1}],4:[function(require,module,exports){
// shim for using process in browser
var process = module.exports = {};
process.nextTick = (function () {
var canSetImmediate = typeof window !== 'undefined'
&& window.setImmediate;
var canPost = typeof window !== 'undefined'
&& window.postMessage && window.addEventListener
;
if (canSetImmediate) {
return function (f) { return window.setImmediate(f) };
}
if (canPost) {
var queue = [];
window.addEventListener('message', function (ev) {
var source = ev.source;
if ((source === window || source === null) && ev.data === 'process-tick') {
ev.stopPropagation();
if (queue.length > 0) {
var fn = queue.shift();
fn();
}
}
}, true);
return function nextTick(fn) {
queue.push(fn);
window.postMessage('process-tick', '*');
};
}
return function nextTick(fn) {
setTimeout(fn, 0);
};
})();
process.title = 'browser';
process.browser = true;
process.env = {};
process.argv = [];
process.binding = function (name) {
throw new Error('process.binding is not supported');
}
// TODO(shtylman)
process.cwd = function () { return '/' };
process.chdir = function (dir) {
throw new Error('process.chdir is not supported');
};
},{}],5:[function(require,module,exports){
/**
* Manage access to data, be it to find, update or remove it
*/
var model = require('./model')
, _ = require('underscore')
;
/**
* Create a new cursor for this collection
* @param {Datastore} db - The datastore this cursor is bound to
* @param {Query} query - The query this cursor will operate on
* @param {Function} execFn - Handler to be executed after cursor has found the results and before the callback passed to find/findOne/update/remove
*/
function Cursor (db, query, execFn) {
this.db = db;
this.query = query || {};
if (execFn) { this.execFn = execFn; }
}
/**
* Set a limit to the number of results
*/
Cursor.prototype.limit = function(limit) {
this._limit = limit;
return this;
};
/**
* Skip a the number of results
*/
Cursor.prototype.skip = function(skip) {
this._skip = skip;
return this;
};
/**
* Sort results of the query
* @param {SortQuery} sortQuery - SortQuery is { field: order }, field can use the dot-notation, order is 1 for ascending and -1 for descending
*/
Cursor.prototype.sort = function(sortQuery) {
this._sort = sortQuery;
return this;
};
/**
* Add the use of a projection
* @param {Object} projection - MongoDB-style projection. {} means take all fields. Then it's { key1: 1, key2: 1 } to take only key1 and key2
* { key1: 0, key2: 0 } to omit only key1 and key2. Except _id, you can't mix takes and omits
*/
Cursor.prototype.projection = function(projection) {
this._projection = projection;
return this;
};
/**
* Apply the projection
*/
Cursor.prototype.project = function (candidates) {
var res = [], self = this
, keepId, action, keys
;
if (this._projection === undefined || Object.keys(this._projection).length === 0) {
return candidates;
}
keepId = this._projection._id === 0 ? false : true;
this._projection = _.omit(this._projection, '_id');
// Check for consistency
keys = Object.keys(this._projection);
keys.forEach(function (k) {
if (action !== undefined && self._projection[k] !== action) { throw new Error("Can't both keep and omit fields except for _id"); }
action = self._projection[k];
});
// Do the actual projection
candidates.forEach(function (candidate) {
var toPush;
if (action === 1) { // pick-type projection
toPush = { $set: {} };
keys.forEach(function (k) {
toPush.$set[k] = model.getDotValue(candidate, k);
if (toPush.$set[k] === undefined) { delete toPush.$set[k]; }
});
toPush = model.modify({}, toPush);
} else { // omit-type projection
toPush = { $unset: {} };
keys.forEach(function (k) { toPush.$unset[k] = true });
toPush = model.modify(candidate, toPush);
}
if (keepId) {
toPush._id = candidate._id;
} else {
delete toPush._id;
}
res.push(toPush);
});
return res;
};
/**
* Get all matching elements
* Will return pointers to matched elements (shallow copies), returning full copies is the role of find or findOne
* This is an internal function, use exec which uses the executor
*
* @param {Function} callback - Signature: err, results
*/
Cursor.prototype._exec = function(_callback) {
var res = [], added = 0, skipped = 0, self = this
, error = null
, i, keys, key
;
function callback (error, res) {
if (self.execFn) {
return self.execFn(error, res, _callback);
} else {
return _callback(error, res);
}
}
this.db.getCandidates(this.query, function (err, candidates) {
if (err) { return callback(err); }
try {
for (i = 0; i < candidates.length; i += 1) {
if (model.match(candidates[i], self.query)) {
// If a sort is defined, wait for the results to be sorted before applying limit and skip
if (!self._sort) {
if (self._skip && self._skip > skipped) {
skipped += 1;
} else {
res.push(candidates[i]);
added += 1;
if (self._limit && self._limit <= added) { break; }
}
} else {
res.push(candidates[i]);
}
}
}
} catch (err) {
return callback(err);
}
// Apply all sorts
if (self._sort) {
keys = Object.keys(self._sort);
// Sorting
var criteria = [];
for (i = 0; i < keys.length; i++) {
key = keys[i];
criteria.push({ key: key, direction: self._sort[key] });
}
res.sort(function(a, b) {
var criterion, compare, i;
for (i = 0; i < criteria.length; i++) {
criterion = criteria[i];
compare = criterion.direction * model.compareThings(model.getDotValue(a, criterion.key), model.getDotValue(b, criterion.key), self.db.compareStrings);
if (compare !== 0) {
return compare;
}
}
return 0;
});
// Applying limit and skip
var limit = self._limit || res.length
, skip = self._skip || 0;
res = res.slice(skip, skip + limit);
}
// Apply projection
try {
res = self.project(res);
} catch (e) {
error = e;
res = undefined;
}
return callback(error, res);
});
};
Cursor.prototype.exec = function () {
this.db.executor.push({ this: this, fn: this._exec, arguments: arguments });
};
// Interface
module.exports = Cursor;
},{"./model":10,"underscore":19}],6:[function(require,module,exports){
/**
* Specific customUtils for the browser, where we don't have access to the Crypto and Buffer modules
*/
/**
* Taken from the crypto-browserify module
* https://github.com/dominictarr/crypto-browserify
* NOTE: Math.random() does not guarantee "cryptographic quality" but we actually don't need it
*/
function randomBytes (size) {
var bytes = new Array(size);
var r;
for (var i = 0, r; i < size; i++) {
if ((i & 0x03) == 0) r = Math.random() * 0x100000000;
bytes[i] = r >>> ((i & 0x03) << 3) & 0xff;
}
return bytes;
}
/**
* Taken from the base64-js module
* https://github.com/beatgammit/base64-js/
*/
function byteArrayToBase64 (uint8) {
var lookup = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/'
, extraBytes = uint8.length % 3 // if we have 1 byte left, pad 2 bytes
, output = ""
, temp, length, i;
function tripletToBase64 (num) {
return lookup[num >> 18 & 0x3F] + lookup[num >> 12 & 0x3F] + lookup[num >> 6 & 0x3F] + lookup[num & 0x3F];
};
// go through the array every three bytes, we'll deal with trailing stuff later
for (i = 0, length = uint8.length - extraBytes; i < length; i += 3) {
temp = (uint8[i] << 16) + (uint8[i + 1] << 8) + (uint8[i + 2]);
output += tripletToBase64(temp);
}
// pad the end with zeros, but make sure to not forget the extra bytes
switch (extraBytes) {
case 1:
temp = uint8[uint8.length - 1];
output += lookup[temp >> 2];
output += lookup[(temp << 4) & 0x3F];
output += '==';
break;
case 2:
temp = (uint8[uint8.length - 2] << 8) + (uint8[uint8.length - 1]);
output += lookup[temp >> 10];
output += lookup[(temp >> 4) & 0x3F];
output += lookup[(temp << 2) & 0x3F];
output += '=';
break;
}
return output;
}
/**
* Return a random alphanumerical string of length len
* There is a very small probability (less than 1/1,000,000) for the length to be less than len
* (il the base64 conversion yields too many pluses and slashes) but
* that's not an issue here
* The probability of a collision is extremely small (need 3*10^12 documents to have one chance in a million of a collision)
* See http://en.wikipedia.org/wiki/Birthday_problem
*/
function uid (len) {
return byteArrayToBase64(randomBytes(Math.ceil(Math.max(8, len * 2)))).replace(/[+\/]/g, '').slice(0, len);
}
module.exports.uid = uid;
},{}],7:[function(require,module,exports){
var customUtils = require('./customUtils')
, model = require('./model')
, async = require('async')
, Executor = require('./executor')
, Index = require('./indexes')
, util = require('util')
, _ = require('underscore')
, Persistence = require('./persistence')
, Cursor = require('./cursor')
;
/**
* Create a new collection
* @param {String} options.filename Optional, datastore will be in-memory only if not provided
* @param {Boolean} options.timestampData Optional, defaults to false. If set to true, createdAt and updatedAt will be created and populated automatically (if not specified by user)
* @param {Boolean} options.inMemoryOnly Optional, defaults to false
* @param {String} options.nodeWebkitAppName Optional, specify the name of your NW app if you want options.filename to be relative to the directory where
* Node Webkit stores application data such as cookies and local storage (the best place to store data in my opinion)
* @param {Boolean} options.autoload Optional, defaults to false
* @param {Function} options.onload Optional, if autoload is used this will be called after the load database with the error object as parameter. If you don't pass it the error will be thrown
* @param {Function} options.afterSerialization/options.beforeDeserialization Optional, serialization hooks
* @param {Number} options.corruptAlertThreshold Optional, threshold after which an alert is thrown if too much data is corrupt
* @param {Function} options.compareStrings Optional, string comparison function that overrides default for sorting
*
* Event Emitter - Events
* * compaction.done - Fired whenever a compaction operation was finished
*/
function Datastore (options) {
var filename;
// Retrocompatibility with v0.6 and before
if (typeof options === 'string') {
filename = options;
this.inMemoryOnly = false; // Default
} else {
options = options || {};
filename = options.filename;
this.inMemoryOnly = options.inMemoryOnly || false;
this.autoload = options.autoload || false;
this.timestampData = options.timestampData || false;
}
// Determine whether in memory or persistent
if (!filename || typeof filename !== 'string' || filename.length === 0) {
this.filename = null;
this.inMemoryOnly = true;
} else {
this.filename = filename;
}
// String comparison function
this.compareStrings = options.compareStrings;
// Persistence handling
this.persistence = new Persistence({ db: this, nodeWebkitAppName: options.nodeWebkitAppName
, afterSerialization: options.afterSerialization
, beforeDeserialization: options.beforeDeserialization
, corruptAlertThreshold: options.corruptAlertThreshold
});
// This new executor is ready if we don't use persistence
// If we do, it will only be ready once loadDatabase is called
this.executor = new Executor();
if (this.inMemoryOnly) { this.executor.ready = true; }
// Indexed by field name, dot notation can be used
// _id is always indexed and since _ids are generated randomly the underlying
// binary is always well-balanced
this.indexes = {};
this.indexes._id = new Index({ fieldName: '_id', unique: true });
this.ttlIndexes = {};
// Queue a load of the database right away and call the onload handler
// By default (no onload handler), if there is an error there, no operation will be possible so warn the user by throwing an exception
if (this.autoload) { this.loadDatabase(options.onload || function (err) {
if (err) { throw err; }
}); }
}
util.inherits(Datastore, require('events').EventEmitter);
/**
* Load the database from the datafile, and trigger the execution of buffered commands if any
*/
Datastore.prototype.loadDatabase = function () {
this.executor.push({ this: this.persistence, fn: this.persistence.loadDatabase, arguments: arguments }, true);
};
/**
* Get an array of all the data in the database
*/
Datastore.prototype.getAllData = function () {
return this.indexes._id.getAll();
};
/**
* Reset all currently defined indexes
*/
Datastore.prototype.resetIndexes = function (newData) {
var self = this;
Object.keys(this.indexes).forEach(function (i) {
self.indexes[i].reset(newData);
});
};
/**
* Ensure an index is kept for this field. Same parameters as lib/indexes
* For now this function is synchronous, we need to test how much time it takes
* We use an async API for consistency with the rest of the code
* @param {String} options.fieldName
* @param {Boolean} options.unique
* @param {Boolean} options.sparse
* @param {Number} options.expireAfterSeconds - Optional, if set this index becomes a TTL index (only works on Date fields, not arrays of Date)
* @param {Function} cb Optional callback, signature: err
*/
Datastore.prototype.ensureIndex = function (options, cb) {
var err
, callback = cb || function () {};
options = options || {};
if (!options.fieldName) {
err = new Error("Cannot create an index without a fieldName");
err.missingFieldName = true;
return callback(err);
}
if (this.indexes[options.fieldName]) { return callback(null); }
this.indexes[options.fieldName] = new Index(options);
if (options.expireAfterSeconds !== undefined) { this.ttlIndexes[options.fieldName] = options.expireAfterSeconds; } // With this implementation index creation is not necessary to ensure TTL but we stick with MongoDB's API here
try {
this.indexes[options.fieldName].insert(this.getAllData());
} catch (e) {
delete this.indexes[options.fieldName];
return callback(e);
}
// We may want to force all options to be persisted including defaults, not just the ones passed the index creation function
this.persistence.persistNewState([{ $$indexCreated: options }], function (err) {
if (err) { return callback(err); }
return callback(null);
});
};
/**
* Remove an index
* @param {String} fieldName
* @param {Function} cb Optional callback, signature: err
*/
Datastore.prototype.removeIndex = function (fieldName, cb) {
var callback = cb || function () {};
delete this.indexes[fieldName];
this.persistence.persistNewState([{ $$indexRemoved: fieldName }], function (err) {
if (err) { return callback(err); }
return callback(null);
});
};
/**
* Add one or several document(s) to all indexes
*/
Datastore.prototype.addToIndexes = function (doc) {
var i, failingIndex, error
, keys = Object.keys(this.indexes)
;
for (i = 0; i < keys.length; i += 1) {
try {
this.indexes[keys[i]].insert(doc);
} catch (e) {
failingIndex = i;
error = e;
break;
}
}
// If an error happened, we need to rollback the insert on all other indexes
if (error) {
for (i = 0; i < failingIndex; i += 1) {
this.indexes[keys[i]].remove(doc);
}
throw error;
}
};
/**
* Remove one or several document(s) from all indexes
*/
Datastore.prototype.removeFromIndexes = function (doc) {
var self = this;
Object.keys(this.indexes).forEach(function (i) {
self.indexes[i].remove(doc);
});
};
/**
* Update one or several documents in all indexes
* To update multiple documents, oldDoc must be an array of { oldDoc, newDoc } pairs
* If one update violates a constraint, all changes are rolled back
*/
Datastore.prototype.updateIndexes = function (oldDoc, newDoc) {
var i, failingIndex, error
, keys = Object.keys(this.indexes)
;
for (i = 0; i < keys.length; i += 1) {
try {
this.indexes[keys[i]].update(oldDoc, newDoc);
} catch (e) {
failingIndex = i;
error = e;
break;
}
}
// If an error happened, we need to rollback the update on all other indexes
if (error) {
for (i = 0; i < failingIndex; i += 1) {
this.indexes[keys[i]].revertUpdate(oldDoc, newDoc);
}
throw error;
}
};
/**
* Return the list of candidates for a given query
* Crude implementation for now, we return the candidates given by the first usable index if any
* We try the following query types, in this order: basic match, $in match, comparison match
* One way to make it better would be to enable the use of multiple indexes if the first usable index
* returns too much data. I may do it in the future.
*
* Returned candidates will be scanned to find and remove all expired documents
*
* @param {Query} query
* @param {Boolean} dontExpireStaleDocs Optional, defaults to false, if true don't remove stale docs. Useful for the remove function which shouldn't be impacted by expirations
* @param {Function} callback Signature err, candidates
*/
Datastore.prototype.getCandidates = function (query, dontExpireStaleDocs, callback) {
var indexNames = Object.keys(this.indexes)
, self = this
, usableQueryKeys;
if (typeof dontExpireStaleDocs === 'function') {
callback = dontExpireStaleDocs;
dontExpireStaleDocs = false;
}
async.waterfall([
// STEP 1: get candidates list by checking indexes from most to least frequent usecase
function (cb) {
// For a basic match
usableQueryKeys = [];
Object.keys(query).forEach(function (k) {
if (typeof query[k] === 'string' || typeof query[k] === 'number' || typeof query[k] === 'boolean' || util.isDate(query[k]) || query[k] === null) {
usableQueryKeys.push(k);
}
});
usableQueryKeys = _.intersection(usableQueryKeys, indexNames);
if (usableQueryKeys.length > 0) {
return cb(null, self.indexes[usableQueryKeys[0]].getMatching(query[usableQueryKeys[0]]));
}
// For a $in match
usableQueryKeys = [];
Object.keys(query).forEach(function (k) {
if (query[k] && query[k].hasOwnProperty('$in')) {
usableQueryKeys.push(k);
}
});
usableQueryKeys = _.intersection(usableQueryKeys, indexNames);
if (usableQueryKeys.length > 0) {
return cb(null, self.indexes[usableQueryKeys[0]].getMatching(query[usableQueryKeys[0]].$in));
}
// For a comparison match
usableQueryKeys = [];
Object.keys(query).forEach(function (k) {
if (query[k] && (query[k].hasOwnProperty('$lt') || query[k].hasOwnProperty('$lte') || query[k].hasOwnProperty('$gt') || query[k].hasOwnProperty('$gte'))) {
usableQueryKeys.push(k);
}
});
usableQueryKeys = _.intersection(usableQueryKeys, indexNames);
if (usableQueryKeys.length > 0) {
return cb(null, self.indexes[usableQueryKeys[0]].getBetweenBounds(query[usableQueryKeys[0]]));
}
// By default, return all the DB data
return cb(null, self.getAllData());
}
// STEP 2: remove all expired documents
, function (docs) {
if (dontExpireStaleDocs) { return callback(null, docs); }
var expiredDocsIds = [], validDocs = [], ttlIndexesFieldNames = Object.keys(self.ttlIndexes);
docs.forEach(function (doc) {
var valid = true;
ttlIndexesFieldNames.forEach(function (i) {
if (doc[i] !== undefined && util.isDate(doc[i]) && Date.now() > doc[i].getTime() + self.ttlIndexes[i] * 1000) {
valid = false;
}
});
if (valid) { validDocs.push(doc); } else { expiredDocsIds.push(doc._id); }
});
async.eachSeries(expiredDocsIds, function (_id, cb) {
self._remove({ _id: _id }, {}, function (err) {
if (err) { return callback(err); }
return cb();
});
}, function (err) {
return callback(null, validDocs);
});
}]);
};
/**
* Insert a new document
* @param {Function} cb Optional callback, signature: err, insertedDoc
*
* @api private Use Datastore.insert which has the same signature
*/
Datastore.prototype._insert = function (newDoc, cb) {
var callback = cb || function () {}
, preparedDoc
;
try {
preparedDoc = this.prepareDocumentForInsertion(newDoc)
this._insertInCache(preparedDoc);
} catch (e) {
return callback(e);
}
this.persistence.persistNewState(util.isArray(preparedDoc) ? preparedDoc : [preparedDoc], function (err) {
if (err) { return callback(err); }
return callback(null, model.deepCopy(preparedDoc));
});
};
/**
* Create a new _id that's not already in use
*/
Datastore.prototype.createNewId = function () {
var tentativeId = customUtils.uid(16);
// Try as many times as needed to get an unused _id. As explained in customUtils, the probability of this ever happening is extremely small, so this is O(1)
if (this.indexes._id.getMatching(tentativeId).length > 0) {
tentativeId = this.createNewId();
}
return tentativeId;
};
/**
* Prepare a document (or array of documents) to be inserted in a database
* Meaning adds _id and timestamps if necessary on a copy of newDoc to avoid any side effect on user input
* @api private
*/
Datastore.prototype.prepareDocumentForInsertion = function (newDoc) {
var preparedDoc, self = this;
if (util.isArray(newDoc)) {
preparedDoc = [];
newDoc.forEach(function (doc) { preparedDoc.push(self.prepareDocumentForInsertion(doc)); });
} else {
preparedDoc = model.deepCopy(newDoc);
if (preparedDoc._id === undefined) { preparedDoc._id = this.createNewId(); }
var now = new Date();
if (this.timestampData && preparedDoc.createdAt === undefined) { preparedDoc.createdAt = now; }
if (this.timestampData && preparedDoc.updatedAt === undefined) { preparedDoc.updatedAt = now; }
model.checkObject(preparedDoc);
}
return preparedDoc;
};
/**
* If newDoc is an array of documents, this will insert all documents in the cache
* @api private
*/
Datastore.prototype._insertInCache = function (preparedDoc) {
if (util.isArray(preparedDoc)) {
this._insertMultipleDocsInCache(preparedDoc);
} else {
this.addToIndexes(preparedDoc);
}
};
/**
* If one insertion fails (e.g. because of a unique constraint), roll back all previous
* inserts and throws the error
* @api private
*/
Datastore.prototype._insertMultipleDocsInCache = function (preparedDocs) {
var i, failingI, error;
for (i = 0; i < preparedDocs.length; i += 1) {
try {
this.addToIndexes(preparedDocs[i]);
} catch (e) {
error = e;
failingI = i;
break;
}
}
if (error) {
for (i = 0; i < failingI; i += 1) {
this.removeFromIndexes(preparedDocs[i]);
}
throw error;
}
};
Datastore.prototype.insert = function () {
this.executor.push({ this: this, fn: this._insert, arguments: arguments });
};
/**
* Count all documents matching the query
* @param {Object} query MongoDB-style query
*/
Datastore.prototype.count = function(query, callback) {
var cursor = new Cursor(this, query, function(err, docs, callback) {
if (err) { return callback(err); }
return callback(null, docs.length);
});
if (typeof callback === 'function') {
cursor.exec(callback);
} else {
return cursor;
}
};
/**
* Find all documents matching the query
* If no callback is passed, we return the cursor so that user can limit, skip and finally exec
* @param {Object} query MongoDB-style query
* @param {Object} projection MongoDB-style projection
*/
Datastore.prototype.find = function (query, projection, callback) {
switch (arguments.length) {
case 1:
projection = {};
// callback is undefined, will return a cursor
break;
case 2:
if (typeof projection === 'function') {
callback = projection;
projection = {};
} // If not assume projection is an object and callback undefined
break;
}
var cursor = new Cursor(this, query, function(err, docs, callback) {
var res = [], i;
if (err) { return callback(err); }
for (i = 0; i < docs.length; i += 1) {
res.push(model.deepCopy(docs[i]));
}
return callback(null, res);
});
cursor.projection(projection);
if (typeof callback === 'function') {
cursor.exec(callback);
} else {
return cursor;
}
};
/**
* Find one document matching the query
* @param {Object} query MongoDB-style query
* @param {Object} projection MongoDB-style projection
*/
Datastore.prototype.findOne = function (query, projection, callback) {
switch (arguments.length) {
case 1:
projection = {};
// callback is undefined, will return a cursor
break;
case 2:
if (typeof projection === 'function') {
callback = projection;
projection = {};
} // If not assume projection is an object and callback undefined
break;
}
var cursor = new Cursor(this, query, function(err, docs, callback) {
if (err) { return callback(err); }
if (docs.length === 1) {
return callback(null, model.deepCopy(docs[0]));
} else {
return callback(null, null);
}
});
cursor.projection(projection).limit(1);
if (typeof callback === 'function') {
cursor.exec(callback);
} else {
return cursor;
}
};
/**
* Update all docs matching query
* @param {Object} query
* @param {Object} updateQuery
* @param {Object} options Optional options
* options.multi If true, can update multiple documents (defaults to false)
* options.upsert If true, document is inserted if the query doesn't match anything
* options.returnUpdatedDocs Defaults to false, if true return as third argument the array of updated matched documents (even if no change actually took place)
* @param {Function} cb Optional callback, signature: (err, numAffected, affectedDocuments, upsert)
* If update was an upsert, upsert flag is set to true
* affectedDocuments can be one of the following:
* * For an upsert, the upserted document
* * For an update with returnUpdatedDocs option false, null
* * For an update with returnUpdatedDocs true and multi false, the updated document
* * For an update with returnUpdatedDocs true and multi true, the array of updated documents
*
* WARNING: The API was changed between v1.7.4 and v1.8, for consistency and readability reasons. Prior and including to v1.7.4,
* the callback signature was (err, numAffected, updated) where updated was the updated document in case of an upsert
* or the array of updated documents for an update if the returnUpdatedDocs option was true. That meant that the type of
* affectedDocuments in a non multi update depended on whether there was an upsert or not, leaving only two ways for the
* user to check whether an upsert had occured: checking the type of affectedDocuments or running another find query on
* the whole dataset to check its size. Both options being ugly, the breaking change was necessary.
*
* @api private Use Datastore.update which has the same signature
*/
Datastore.prototype._update = function (query, updateQuery, options, cb) {
var callback
, self = this
, numReplaced = 0
, multi, upsert
, i
;
if (typeof options === 'function') { cb = options; options = {}; }
callback = cb || function () {};
multi = options.multi !== undefined ? options.multi : false;
upsert = options.upsert !== undefined ? options.upsert : false;
async.waterfall([
function (cb) { // If upsert option is set, check whether we need to insert the doc
if (!upsert) { return cb(); }
// Need to use an internal function not tied to the executor to avoid deadlock
var cursor = new Cursor(self, query);
cursor.limit(1)._exec(function (err, docs) {
if (err) { return callback(err); }
if (docs.length === 1) {
return cb();
} else {
var toBeInserted;
try {
model.checkObject(updateQuery);
// updateQuery is a simple object with no modifier, use it as the document to insert
toBeInserted = updateQuery;
} catch (e) {
// updateQuery contains modifiers, use the find query as the base,
// strip it from all operators and update it according to updateQuery
try {
toBeInserted = model.modify(model.deepCopy(query, true), updateQuery);
} catch (err) {
return callback(err);
}
}
return self._insert(toBeInserted, function (err, newDoc) {
if (err) { return callback(err); }
return callback(null, 1, newDoc, true);
});
}
});
}
, function () { // Perform the update
var modifiedDoc , modifications = [], createdAt;
self.getCandidates(query, function (err, candidates) {
if (err) { return callback(err); }
// Preparing update (if an error is thrown here neither the datafile nor
// the in-memory indexes are affected)
try {
for (i = 0; i < candidates.length; i += 1) {
if (model.match(candidates[i], query) && (multi || numReplaced === 0)) {
numReplaced += 1;
if (self.timestampData) { createdAt = candidates[i].createdAt; }
modifiedDoc = model.modify(candidates[i], updateQuery);
if (self.timestampData) {
modifiedDoc.createdAt = createdAt;
modifiedDoc.updatedAt = new Date();
}
modifications.push({ oldDoc: candidates[i], newDoc: modifiedDoc });
}
}
} catch (err) {
return callback(err);
}
// Change the docs in memory
try {
self.updateIndexes(modifications);
} catch (err) {
return callback(err);
}
// Update the datafile
var updatedDocs = _.pluck(modifications, 'newDoc');
self.persistence.persistNewState(updatedDocs, function (err) {
if (err) { return callback(err); }
if (!options.returnUpdatedDocs) {
return callback(null, numReplaced);
} else {
var updatedDocsDC = [];
updatedDocs.forEach(function (doc) { updatedDocsDC.push(model.deepCopy(doc)); });
if (! multi) { updatedDocsDC = updatedDocsDC[0]; }
return callback(null, numReplaced, updatedDocsDC);
}
});
});
}]);
};
Datastore.prototype.update = function () {
this.executor.push({ this: this, fn: this._update, arguments: arguments });
};
/**
* Remove all docs matching the query
* For now very naive implementation (similar to update)
* @param {Object} query
* @param {Object} options Optional options
* options.multi If true, can update multiple documents (defaults to false)
* @param {Function} cb Optional callback, signature: err, numRemoved
*
* @api private Use Datastore.remove which has the same signature
*/
Datastore.prototype._remove = function (query, options, cb) {
var callback
, self = this, numRemoved = 0, removedDocs = [], multi
;
if (typeof options === 'function') { cb = options; options = {}; }
callback = cb || function () {};
multi = options.multi !== undefined ? options.multi : false;
this.getCandidates(query, true, function (err, candidates) {
if (err) { return callback(err); }
try {
candidates.forEach(function (d) {
if (model.match(d, query) && (multi || numRemoved === 0)) {
numRemoved += 1;
removedDocs.push({ $$deleted: true, _id: d._id });
self.removeFromIndexes(d);
}
});
} catch (err) { return callback(err); }
self.persistence.persistNewState(removedDocs, function (err) {
if (err) { return callback(err); }
return callback(null, numRemoved);
});
});
};
Datastore.prototype.remove = function () {
this.executor.push({ this: this, fn: this._remove, arguments: arguments });
};
module.exports = Datastore;
},{"./cursor":5,"./customUtils":6,"./executor":8,"./indexes":9,"./model":10,"./persistence":11,"async":13,"events":1,"underscore":19,"util":3}],8:[function(require,module,exports){
var process=require("__browserify_process");/**
* Responsible for sequentially executing actions on the database
*/
var async = require('async')
;
function Executor () {
this.buffer = [];
this.ready = false;
// This queue will execute all commands, one-by-one in order
this.queue = async.queue(function (task, cb) {
var newArguments = [];
// task.arguments is an array-like object on which adding a new field doesn't work, so we transform it into a real array
for (var i = 0; i < task.arguments.length; i += 1) { newArguments.push(task.arguments[i]); }
var lastArg = task.arguments[task.arguments.length - 1];
// Always tell the queue task is complete. Execute callback if any was given.
if (typeof lastArg === 'function') {
// Callback was supplied
newArguments[newArguments.length - 1] = function () {
if (typeof setImmediate === 'function') {
setImmediate(cb);
} else {
process.nextTick(cb);
}
lastArg.apply(null, arguments);
};
} else if (!lastArg && task.arguments.length !== 0) {
// false/undefined/null supplied as callbback
newArguments[newArguments.length - 1] = function () { cb(); };
} else {
// Nothing supplied as callback
newArguments.push(function () { cb(); });
}
task.fn.apply(task.this, newArguments);
}, 1);
}
/**
* If executor is ready, queue task (and process it immediately if executor was idle)
* If not, buffer task for later processing
* @param {Object} task
* task.this - Object to use as this
* task.fn - Function to execute
* task.arguments - Array of arguments, IMPORTANT: only the last argument may be a function (the callback)
* and the last argument cannot be false/undefined/null
* @param {Boolean} forceQueuing Optional (defaults to false) force executor to queue task even if it is not ready
*/
Executor.prototype.push = function (task, forceQueuing) {
if (this.ready || forceQueuing) {
this.queue.push(task);
} else {
this.buffer.push(task);
}
};
/**
* Queue all tasks in buffer (in the same order they came in)
* Automatically sets executor as ready
*/
Executor.prototype.processBuffer = function () {
var i;
this.ready = true;
for (i = 0; i < this.buffer.length; i += 1) { this.queue.push(this.buffer[i]); }
this.buffer = [];
};
// Interface
module.exports = Executor;
},{"__browserify_process":4,"async":13}],9:[function(require,module,exports){
var BinarySearchTree = require('binary-search-tree').AVLTree
, model = require('./model')
, _ = require('underscore')
, util = require('util')
;
/**
* Two indexed pointers are equal iif they point to the same place
*/
function checkValueEquality (a, b) {
return a === b;
}
/**
* Type-aware projection
*/
function projectForUnique (elt) {
if (elt === null) { return '$null'; }
if (typeof elt === 'string') { return '$string' + elt; }
if (typeof elt === 'boolean') { return '$boolean' + elt; }
if (typeof elt === 'number') { return '$number' + elt; }
if (util.isArray(elt)) { return '$date' + elt.getTime(); }
return elt; // Arrays and objects, will check for pointer equality
}
/**
* Create a new index
* All methods on an index guarantee that either the whole operation was successful and the index changed
* or the operation was unsuccessful and an error is thrown while the index is unchanged
* @param {String} options.fieldName On which field should the index apply (can use dot notation to index on sub fields)
* @param {Boolean} options.unique Optional, enforce a unique constraint (default: false)
* @param {Boolean} options.sparse Optional, allow a sparse index (we can have documents for which fieldName is undefined) (default: false)
*/
function Index (options) {
this.fieldName = options.fieldName;
this.unique = options.unique || false;
this.sparse = options.sparse || false;
this.treeOptions = { unique: this.unique, compareKeys: model.compareThings, checkValueEquality: checkValueEquality };
this.reset(); // No data in the beginning
}
/**
* Reset an index
* @param {Document or Array of documents} newData Optional, data to initialize the index with
* If an error is thrown during insertion, the index is not modified
*/
Index.prototype.reset = function (newData) {
this.tree = new BinarySearchTree(this.treeOptions);
if (newData) { this.insert(newData); }
};
/**
* Insert a new document in the index
* If an array is passed, we insert all its elements (if one insertion fails the index is not modified)
* O(log(n))
*/
Index.prototype.insert = function (doc) {
var key, self = this
, keys, i, failingI, error
;
if (util.isArray(doc)) { this.insertMultipleDocs(doc); return; }
key = model.getDotValue(doc, this.fieldName);
// We don't index documents that don't contain the field if the index is sparse
if (key === undefined && this.sparse) { return; }
if (!util.isArray(key)) {
this.tree.insert(key, doc);
} else {
// If an insert fails due to a unique constraint, roll back all inserts before it
keys = _.uniq(key, projectForUnique);
for (i = 0; i < keys.length; i += 1) {
try {
this.tree.insert(keys[i], doc);
} catch (e) {
error = e;
failingI = i;
break;
}
}
if (error) {
for (i = 0; i < failingI; i += 1) {
this.tree.delete(keys[i], doc);
}
throw error;
}
}
};
/**
* Insert an array of documents in the index
* If a constraint is violated, the changes should be rolled back and an error thrown
*
* @API private
*/
Index.prototype.insertMultipleDocs = function (docs) {
var i, error, failingI;
for (i = 0; i < docs.length; i += 1) {
try {
this.insert(docs[i]);
} catch (e) {
error = e;
failingI = i;
break;
}
}
if (error) {
for (i = 0; i < failingI; i += 1) {
this.remove(docs[i]);
}
throw error;
}
};
/**
* Remove a document from the index
* If an array is passed, we remove all its elements
* The remove operation is safe with regards to the 'unique' constraint
* O(log(n))
*/
Index.prototype.remove = function (doc) {
var key, self = this;
if (util.isArray(doc)) { doc.forEach(function (d) { self.remove(d); }); return; }
key = model.getDotValue(doc, this.fieldName);
if (key === undefined && this.sparse) { return; }
if (!util.isArray(key)) {
this.tree.delete(key, doc);
} else {
_.uniq(key, projectForUnique).forEach(function (_key) {
self.tree.delete(_key, doc);
});
}
};
/**
* Update a document in the index
* If a constraint is violated, changes are rolled back and an error thrown
* Naive implementation, still in O(log(n))
*/
Index.prototype.update = function (oldDoc, newDoc) {
if (util.isArray(oldDoc)) { this.updateMultipleDocs(oldDoc); return; }
this.remove(oldDoc);
try {
this.insert(newDoc);
} catch (e) {
this.insert(oldDoc);
throw e;
}
};
/**
* Update multiple documents in the index
* If a constraint is violated, the changes need to be rolled back
* and an error thrown
* @param {Array of oldDoc, newDoc pairs} pairs
*
* @API private
*/
Index.prototype.updateMultipleDocs = function (pairs) {
var i, failingI, error;
for (i = 0; i < pairs.length; i += 1) {
this.remove(pairs[i].oldDoc);
}
for (i = 0; i < pairs.length; i += 1) {
try {
this.insert(pairs[i].newDoc);
} catch (e) {
error = e;
failingI = i;
break;
}
}
// If an error was raised, roll back changes in the inverse order
if (error) {
for (i = 0; i < failingI; i += 1) {
this.remove(pairs[i].newDoc);
}
for (i = 0; i < pairs.length; i += 1) {
this.insert(pairs[i].oldDoc);
}
throw error;
}
};
/**
* Revert an update
*/
Index.prototype.revertUpdate = function (oldDoc, newDoc) {
var revert = [];
if (!util.isArray(oldDoc)) {
this.update(newDoc, oldDoc);
} else {
oldDoc.forEach(function (pair) {
revert.push({ oldDoc: pair.newDoc, newDoc: pair.oldDoc });
});
this.update(revert);
}
};
/**
* Get all documents in index whose key match value (if it is a Thing) or one of the elements of value (if it is an array of Things)
* @param {Thing} value Value to match the key against
* @return {Array of documents}
*/
Index.prototype.getMatching = function (value) {
var self = this;
if (!util.isArray(value)) {
return self.tree.search(value);
} else {
var _res = {}, res = [];
value.forEach(function (v) {
self.getMatching(v).forEach(function (doc) {
_res[doc._id] = doc;
});
});
Object.keys(_res).forEach(function (_id) {
res.push(_res[_id]);
});
return res;
}
};
/**
* Get all documents in index whose key is between bounds are they are defined by query
* Documents are sorted by key
* @param {Query} query
* @return {Array of documents}
*/
Index.prototype.getBetweenBounds = function (query) {
return this.tree.betweenBounds(query);
};
/**
* Get all elements in the index
* @return {Array of documents}
*/
Index.prototype.getAll = function () {
var res = [];
this.tree.executeOnEveryNode(function (node) {
var i;
for (i = 0; i < node.data.length; i += 1) {
res.push(node.data[i]);
}
});
return res;
};
// Interface
module.exports = Index;
},{"./model":10,"binary-search-tree":14,"underscore":19,"util":3}],10:[function(require,module,exports){
/**
* Handle models (i.e. docs)
* Serialization/deserialization
* Copying
* Querying, update
*/
var util = require('util')
, _ = require('underscore')
, modifierFunctions = {}
, lastStepModifierFunctions = {}
, comparisonFunctions = {}
, logicalOperators = {}
, arrayComparisonFunctions = {}
;
/**
* Check a key, throw an error if the key is non valid
* @param {String} k key
* @param {Model} v value, needed to treat the Date edge case
* Non-treatable edge cases here: if part of the object if of the form { $$date: number } or { $$deleted: true }
* Its serialized-then-deserialized version it will transformed into a Date object
* But you really need to want it to trigger such behaviour, even when warned not to use '$' at the beginning of the field names...
*/
function checkKey (k, v) {
if (typeof k === 'number') {
k = k.toString();
}
if (k[0] === '$' && !(k === '$$date' && typeof v === 'number') && !(k === '$$deleted' && v === true) && !(k === '$$indexCreated') && !(k === '$$indexRemoved')) {
throw new Error('Field names cannot begin with the $ character');
}
if (k.indexOf('.') !== -1) {
throw new Error('Field names cannot contain a .');
}
}
/**
* Check a DB object and throw an error if it's not valid
* Works by applying the above checkKey function to all fields recursively
*/
function checkObject (obj) {
if (util.isArray(obj)) {
obj.forEach(function (o) {
checkObject(o);
});
}
if (typeof obj === 'object' && obj !== null) {
Object.keys(obj).forEach(function (k) {
checkKey(k, obj[k]);
checkObject(obj[k]);
});
}
}
/**
* Serialize an object to be persisted to a one-line string
* For serialization/deserialization, we use the native JSON parser and not eval or Function
* That gives us less freedom but data entered in the database may come from users
* so eval and the like are not safe
* Accepted primitive types: Number, String, Boolean, Date, null
* Accepted secondary types: Objects, Arrays
*/
function serialize (obj) {
var res;
res = JSON.stringify(obj, function (k, v) {
checkKey(k, v);
if (v === undefined) { return undefined; }
if (v === null) { return null; }
// Hackish way of checking if object is Date (this way it works between execution contexts in node-webkit).
// We can't use value directly because for dates it is already string in this function (date.toJSON was already called), so we use this
if (typeof this[k].getTime === 'function') { return { $$date: this[k].getTime() }; }
return v;
});
return res;
}
/**
* From a one-line representation of an object generate by the serialize function
* Return the object itself
*/
function deserialize (rawData) {
return JSON.parse(rawData, function (k, v) {
if (k === '$$date') { return new Date(v); }
if (typeof v === 'string' || typeof v === 'number' || typeof v === 'boolean' || v === null) { return v; }
if (v && v.$$date) { return v.$$date; }
return v;
});
}
/**
* Deep copy a DB object
* The optional strictKeys flag (defaulting to false) indicates whether to copy everything or only fields
* where the keys are valid, i.e. don't begin with $ and don't contain a .
*/
function deepCopy (obj, strictKeys) {
var res;
if ( typeof obj === 'boolean' ||
typeof obj === 'number' ||
typeof obj === 'string' ||
obj === null ||
(util.isDate(obj)) ) {
return obj;
}
if (util.isArray(obj)) {
res = [];
obj.forEach(function (o) { res.push(deepCopy(o, strictKeys)); });
return res;
}
if (typeof obj === 'object') {
res = {};
Object.keys(obj).forEach(function (k) {
if (!strictKeys || (k[0] !== '$' && k.indexOf('.') === -1)) {
res[k] = deepCopy(obj[k], strictKeys);
}
});
return res;
}
return undefined; // For now everything else is undefined. We should probably throw an error instead
}
/**
* Tells if an object is a primitive type or a "real" object
* Arrays are considered primitive
*/
function isPrimitiveType (obj) {
return ( typeof obj === 'boolean' ||
typeof obj === 'number' ||
typeof obj === 'string' ||
obj === null ||
util.isDate(obj) ||
util.isArray(obj));
}
/**
* Utility functions for comparing things
* Assumes type checking was already done (a and b already have the same type)
* compareNSB works for numbers, strings and booleans
*/
function compareNSB (a, b) {
if (a < b) { return -1; }
if (a > b) { return 1; }
return 0;
}
function compareArrays (a, b) {
var i, comp;
for (i = 0; i < Math.min(a.length, b.length); i += 1) {
comp = compareThings(a[i], b[i]);
if (comp !== 0) { return comp; }
}
// Common section was identical, longest one wins
return compareNSB(a.length, b.length);
}
/**
* Compare { things U undefined }
* Things are defined as any native types (string, number, boolean, null, date) and objects
* We need to compare with undefined as it will be used in indexes
* In the case of objects and arrays, we deep-compare
* If two objects dont have the same type, the (arbitrary) type hierarchy is: undefined, null, number, strings, boolean, dates, arrays, objects
* Return -1 if a < b, 1 if a > b and 0 if a = b (note that equality here is NOT the same as defined in areThingsEqual!)
*
* @param {Function} _compareStrings String comparing function, returning -1, 0 or 1, overriding default string comparison (useful for languages with accented letters)
*/
function compareThings (a, b, _compareStrings) {
var aKeys, bKeys, comp, i
, compareStrings = _compareStrings || compareNSB;
// undefined
if (a === undefined) { return b === undefined ? 0 : -1; }
if (b === undefined) { return a === undefined ? 0 : 1; }
// null
if (a === null) { return b === null ? 0 : -1; }
if (b === null) { return a === null ? 0 : 1; }
// Numbers
if (typeof a === 'number') { return typeof b === 'number' ? compareNSB(a, b) : -1; }
if (typeof b === 'number') { return typeof a === 'number' ? compareNSB(a, b) : 1; }
// Strings
if (typeof a === 'string') { return typeof b === 'string' ? compareStrings(a, b) : -1; }
if (typeof b === 'string') { return typeof a === 'string' ? compareStrings(a, b) : 1; }
// Booleans
if (typeof a === 'boolean') { return typeof b === 'boolean' ? compareNSB(a, b) : -1; }
if (typeof b === 'boolean') { return typeof a === 'boolean' ? compareNSB(a, b) : 1; }
// Dates
if (util.isDate(a)) { return util.isDate(b) ? compareNSB(a.getTime(), b.getTime()) : -1; }
if (util.isDate(b)) { return util.isDate(a) ? compareNSB(a.getTime(), b.getTime()) : 1; }
// Arrays (first element is most significant and so on)
if (util.isArray(a)) { return util.isArray(b) ? compareArrays(a, b) : -1; }
if (util.isArray(b)) { return util.isArray(a) ? compareArrays(a, b) : 1; }
// Objects
aKeys = Object.keys(a).sort();
bKeys = Object.keys(b).sort();
for (i = 0; i < Math.min(aKeys.length, bKeys.length); i += 1) {
comp = compareThings(a[aKeys[i]], b[bKeys[i]]);
if (comp !== 0) { return comp; }
}
return compareNSB(aKeys.length, bKeys.length);
}
// ==============================================================
// Updating documents
// ==============================================================
/**
* The signature of modifier functions is as follows
* Their structure is always the same: recursively follow the dot notation while creating
* the nested documents if needed, then apply the "last step modifier"
* @param {Object} obj The model to modify
* @param {String} field Can contain dots, in that case that means we will set a subfield recursively
* @param {Model} value
*/
/**
* Set a field to a new value
*/
lastStepModifierFunctions.$set = function (obj, field, value) {
obj[field] = value;
};
/**
* Unset a field
*/
lastStepModifierFunctions.$unset = function (obj, field, value) {
delete obj[field];
};
/**
* Push an element to the end of an array field
* Optional modifier $each instead of value to push several values
* Optional modifier $slice to slice the resulting array, see https://docs.mongodb.org/manual/reference/operator/update/slice/
* Différeence with MongoDB: if $slice is specified and not $each, we act as if value is an empty array
*/
lastStepModifierFunctions.$push = function (obj, field, value) {
// Create the array if it doesn't exist
if (!obj.hasOwnProperty(field)) { obj[field] = []; }
if (!util.isArray(obj[field])) { throw new Error("Can't $push an element on non-array values"); }
if (value !== null && typeof value === 'object' && value.$slice && value.$each === undefined) {
value.$each = [];
}
if (value !== null && typeof value === 'object' && value.$each) {
if (Object.keys(value).length >= 3 || (Object.keys(value).length === 2 && value.$slice === undefined)) { throw new Error("Can only use $slice in cunjunction with $each when $push to array"); }
if (!util.isArray(value.$each)) { throw new Error("$each requires an array value"); }
value.$each.forEach(function (v) {
obj[field].push(v);
});
if (value.$slice === undefined || typeof value.$slice !== 'number') { return; }
if (value.$slice === 0) {
obj[field] = [];
} else {
var start, end, n = obj[field].length;
if (value.$slice < 0) {
start = Math.max(0, n + value.$slice);
end = n;
} else if (value.$slice > 0) {
start = 0;
end = Math.min(n, value.$slice);
}
obj[field] = obj[field].slice(start, end);
}
} else {
obj[field].push(value);
}
};
/**
* Add an element to an array field only if it is not already in it
* No modification if the element is already in the array
* Note that it doesn't check whether the original array contains duplicates
*/
lastStepModifierFunctions.$addToSet = function (obj, field, value) {
var addToSet = true;
// Create the array if it doesn't exist
if (!obj.hasOwnProperty(field)) { obj[field] = []; }
if (!util.isArray(obj[field])) { throw new Error("Can't $addToSet an element on non-array values"); }
if (value !== null && typeof value === 'object' && value.$each) {
if (Object.keys(value).length > 1) { throw new Error("Can't use another field in conjunction with $each"); }
if (!util.isArray(value.$each)) { throw new Error("$each requires an array value"); }
value.$each.forEach(function (v) {
lastStepModifierFunctions.$addToSet(obj, field, v);
});
} else {
obj[field].forEach(function (v) {
if (compareThings(v, value) === 0) { addToSet = false; }
});
if (addToSet) { obj[field].push(value); }
}
};
/**
* Remove the first or last element of an array
*/
lastStepModifierFunctions.$pop = function (obj, field, value) {
if (!util.isArray(obj[field])) { throw new Error("Can't $pop an element from non-array values"); }
if (typeof value !== 'number') { throw new Error(value + " isn't an integer, can't use it with $pop"); }
if (value === 0) { return; }
if (value > 0) {
obj[field] = obj[field].slice(0, obj[field].length - 1);
} else {
obj[field] = obj[field].slice(1);
}
};
/**
* Removes all instances of a value from an existing array
*/
lastStepModifierFunctions.$pull = function (obj, field, value) {
var arr, i;
if (!util.isArray(obj[field])) { throw new Error("Can't $pull an element from non-array values"); }
arr = obj[field];
for (i = arr.length - 1; i >= 0; i -= 1) {
if (match(arr[i], value)) {
arr.splice(i, 1);
}
}
};
/**
* Increment a numeric field's value
*/
lastStepModifierFunctions.$inc = function (obj, field, value) {
if (typeof value !== 'number') { throw new Error(value + " must be a number"); }
if (typeof obj[field] !== 'number') {
if (!_.has(obj, field)) {
obj[field] = value;
} else {
throw new Error("Don't use the $inc modifier on non-number fields");
}
} else {
obj[field] += value;
}
};
/**
* Updates the value of the field, only if specified field is greater than the current value of the field
*/
lastStepModifierFunctions.$max = function (obj, field, value) {
if (typeof obj[field] === 'undefined') {
obj[field] = value;
} else if (value > obj[field]) {
obj[field] = value;
}
};
/**
* Updates the value of the field, only if specified field is smaller than the current value of the field
*/
lastStepModifierFunctions.$min = function (obj, field, value) {
if (typeof obj[field] === 'undefined') {
obj[field] = value;
} else if (value < obj[field]) {
obj[field] = value;
}
};
// Given its name, create the complete modifier function
function createModifierFunction (modifier) {
return function (obj, field, value) {
var fieldParts = typeof field === 'string' ? field.split('.') : field;
if (fieldParts.length === 1) {
lastStepModifierFunctions[modifier](obj, field, value);
} else {
if (obj[fieldParts[0]] === undefined) {
if (modifier === '$unset') { return; } // Bad looking specific fix, needs to be generalized modifiers that behave like $unset are implemented
obj[fieldParts[0]] = {};
}
modifierFunctions[modifier](obj[fieldParts[0]], fieldParts.slice(1), value);
}
};
}
// Actually create all modifier functions
Object.keys(lastStepModifierFunctions).forEach(function (modifier) {
modifierFunctions[modifier] = createModifierFunction(modifier);
});
/**
* Modify a DB object according to an update query
*/
function modify (obj, updateQuery) {
var keys = Object.keys(updateQuery)
, firstChars = _.map(keys, function (item) { return item[0]; })
, dollarFirstChars = _.filter(firstChars, function (c) { return c === '$'; })
, newDoc, modifiers
;
if (keys.indexOf('_id') !== -1 && updateQuery._id !== obj._id) { throw new Error("You cannot change a document's _id"); }
if (dollarFirstChars.length !== 0 && dollarFirstChars.length !== firstChars.length) {
throw new Error("You cannot mix modifiers and normal fields");
}
if (dollarFirstChars.length === 0) {
// Simply replace the object with the update query contents
newDoc = deepCopy(updateQuery);
newDoc._id = obj._id;
} else {
// Apply modifiers
modifiers = _.uniq(keys);
newDoc = deepCopy(obj);
modifiers.forEach(function (m) {
var keys;
if (!modifierFunctions[m]) { throw new Error("Unknown modifier " + m); }
// Can't rely on Object.keys throwing on non objects since ES6
// Not 100% satisfying as non objects can be interpreted as objects but no false negatives so we can live with it
if (typeof updateQuery[m] !== 'object') {
throw new Error("Modifier " + m + "'s argument must be an object");
}
keys = Object.keys(updateQuery[m]);
keys.forEach(function (k) {
modifierFunctions[m](newDoc, k, updateQuery[m][k]);
});
});
}
// Check result is valid and return it
checkObject(newDoc);
if (obj._id !== newDoc._id) { throw new Error("You can't change a document's _id"); }
return newDoc;
};
// ==============================================================
// Finding documents
// ==============================================================
/**
* Get a value from object with dot notation
* @param {Object} obj
* @param {String} field
*/
function getDotValue (obj, field) {
var fieldParts = typeof field === 'string' ? field.split('.') : field
, i, objs;
if (!obj) { return undefined; } // field cannot be empty so that means we should return undefined so that nothing can match
if (fieldParts.length === 0) { return obj; }
if (fieldParts.length === 1) { return obj[fieldParts[0]]; }
if (util.isArray(obj[fieldParts[0]])) {
// If the next field is an integer, return only this item of the array
i = parseInt(fieldParts[1], 10);
if (typeof i === 'number' && !isNaN(i)) {
return getDotValue(obj[fieldParts[0]][i], fieldParts.slice(2))
}
// Return the array of values
objs = new Array();
for (i = 0; i < obj[fieldParts[0]].length; i += 1) {
objs.push(getDotValue(obj[fieldParts[0]][i], fieldParts.slice(1)));
}
return objs;
} else {
return getDotValue(obj[fieldParts[0]], fieldParts.slice(1));
}
}
/**
* Check whether 'things' are equal
* Things are defined as any native types (string, number, boolean, null, date) and objects
* In the case of object, we check deep equality
* Returns true if they are, false otherwise
*/
function areThingsEqual (a, b) {
var aKeys , bKeys , i;
// Strings, booleans, numbers, null
if (a === null || typeof a === 'string' || typeof a === 'boolean' || typeof a === 'number' ||
b === null || typeof b === 'string' || typeof b === 'boolean' || typeof b === 'number') { return a === b; }
// Dates
if (util.isDate(a) || util.isDate(b)) { return util.isDate(a) && util.isDate(b) && a.getTime() === b.getTime(); }
// Arrays (no match since arrays are used as a $in)
// undefined (no match since they mean field doesn't exist and can't be serialized)
if ((!(util.isArray(a) && util.isArray(b)) && (util.isArray(a) || util.isArray(b))) || a === undefined || b === undefined) { return false; }
// General objects (check for deep equality)
// a and b should be objects at this point
try {
aKeys = Object.keys(a);
bKeys = Object.keys(b);
} catch (e) {
return false;
}
if (aKeys.length !== bKeys.length) { return false; }
for (i = 0; i < aKeys.length; i += 1) {
if (bKeys.indexOf(aKeys[i]) === -1) { return false; }
if (!areThingsEqual(a[aKeys[i]], b[aKeys[i]])) { return false; }
}
return true;
}
/**
* Check that two values are comparable
*/
function areComparable (a, b) {
if (typeof a !== 'string' && typeof a !== 'number' && !util.isDate(a) &&
typeof b !== 'string' && typeof b !== 'number' && !util.isDate(b)) {
return false;
}
if (typeof a !== typeof b) { return false; }
return true;
}
/**
* Arithmetic and comparison operators
* @param {Native value} a Value in the object
* @param {Native value} b Value in the query
*/
comparisonFunctions.$lt = function (a, b) {
return areComparable(a, b) && a < b;
};
comparisonFunctions.$lte = function (a, b) {
return areComparable(a, b) && a <= b;
};
comparisonFunctions.$gt = function (a, b) {
return areComparable(a, b) && a > b;
};
comparisonFunctions.$gte = function (a, b) {
return areComparable(a, b) && a >= b;
};
comparisonFunctions.$ne = function (a, b) {
if (a === undefined) { return true; }
return !areThingsEqual(a, b);
};
comparisonFunctions.$in = function (a, b) {
var i;
if (!util.isArray(b)) { throw new Error("$in operator called with a non-array"); }
for (i = 0; i < b.length; i += 1) {
if (areThingsEqual(a, b[i])) { return true; }
}
return false;
};
comparisonFunctions.$nin = function (a, b) {
if (!util.isArray(b)) { throw new Error("$nin operator called with a non-array"); }
return !comparisonFunctions.$in(a, b);
};
comparisonFunctions.$regex = function (a, b) {
if (!util.isRegExp(b)) { throw new Error("$regex operator called with non regular expression"); }
if (typeof a !== 'string') {
return false
} else {
return b.test(a);
}
};
comparisonFunctions.$exists = function (value, exists) {
if (exists || exists === '') { // This will be true for all values of exists except false, null, undefined and 0
exists = true; // That's strange behaviour (we should only use true/false) but that's the way Mongo does it...
} else {
exists = false;
}
if (value === undefined) {
return !exists
} else {
return exists;
}
};
// Specific to arrays
comparisonFunctions.$size = function (obj, value) {
if (!util.isArray(obj)) { return false; }
if (value % 1 !== 0) { throw new Error("$size operator called without an integer"); }
return (obj.length == value);
};
comparisonFunctions.$elemMatch = function (obj, value) {
if (!util.isArray(obj)) { return false; }
var i = obj.length;
var result = false; // Initialize result
while (i--) {
if (match(obj[i], value)) { // If match for array element, return true
result = true;
break;
}
}
return result;
};
arrayComparisonFunctions.$size = true;
arrayComparisonFunctions.$elemMatch = true;
/**
* Match any of the subqueries
* @param {Model} obj
* @param {Array of Queries} query
*/
logicalOperators.$or = function (obj, query) {
var i;
if (!util.isArray(query)) { throw new Error("$or operator used without an array"); }
for (i = 0; i < query.length; i += 1) {
if (match(obj, query[i])) { return true; }
}
return false;
};
/**
* Match all of the subqueries
* @param {Model} obj
* @param {Array of Queries} query
*/
logicalOperators.$and = function (obj, query) {
var i;
if (!util.isArray(query)) { throw new Error("$and operator used without an array"); }
for (i = 0; i < query.length; i += 1) {
if (!match(obj, query[i])) { return false; }
}
return true;
};
/**
* Inverted match of the query
* @param {Model} obj
* @param {Query} query
*/
logicalOperators.$not = function (obj, query) {
return !match(obj, query);
};
/**
* Use a function to match
* @param {Model} obj
* @param {Query} query
*/
logicalOperators.$where = function (obj, fn) {
var result;
if (!_.isFunction(fn)) { throw new Error("$where operator used without a function"); }
result = fn.call(obj);
if (!_.isBoolean(result)) { throw new Error("$where function must return boolean"); }
return result;
};
/**
* Tell if a given document matches a query
* @param {Object} obj Document to check
* @param {Object} query
*/
function match (obj, query) {
var queryKeys, queryKey, queryValue, i;
// Primitive query against a primitive type
// This is a bit of a hack since we construct an object with an arbitrary key only to dereference it later
// But I don't have time for a cleaner implementation now
if (isPrimitiveType(obj) || isPrimitiveType(query)) {
return matchQueryPart({ needAKey: obj }, 'needAKey', query);
}
// Normal query
queryKeys = Object.keys(query);
for (i = 0; i < queryKeys.length; i += 1) {
queryKey = queryKeys[i];
queryValue = query[queryKey];
if (queryKey[0] === '$') {
if (!logicalOperators[queryKey]) { throw new Error("Unknown logical operator " + queryKey); }
if (!logicalOperators[queryKey](obj, queryValue)) { return false; }
} else {
if (!matchQueryPart(obj, queryKey, queryValue)) { return false; }
}
}
return true;
};
/**
* Match an object against a specific { key: value } part of a query
* if the treatObjAsValue flag is set, don't try to match every part separately, but the array as a whole
*/
function matchQueryPart (obj, queryKey, queryValue, treatObjAsValue) {
var objValue = getDotValue(obj, queryKey)
, i, keys, firstChars, dollarFirstChars;
// Check if the value is an array if we don't force a treatment as value
if (util.isArray(objValue) && !treatObjAsValue) {
// If the queryValue is an array, try to perform an exact match
if (util.isArray(queryValue)) {
return matchQueryPart(obj, queryKey, queryValue, true);
}
// Check if we are using an array-specific comparison function
if (queryValue !== null && typeof queryValue === 'object' && !util.isRegExp(queryValue)) {
keys = Object.keys(queryValue);
for (i = 0; i < keys.length; i += 1) {
if (arrayComparisonFunctions[keys[i]]) { return matchQueryPart(obj, queryKey, queryValue, true); }
}
}
// If not, treat it as an array of { obj, query } where there needs to be at least one match
for (i = 0; i < objValue.length; i += 1) {
if (matchQueryPart({ k: objValue[i] }, 'k', queryValue)) { return true; } // k here could be any string
}
return false;
}
// queryValue is an actual object. Determine whether it contains comparison operators
// or only normal fields. Mixed objects are not allowed
if (queryValue !== null && typeof queryValue === 'object' && !util.isRegExp(queryValue) && !util.isArray(queryValue)) {
keys = Object.keys(queryValue);
firstChars = _.map(keys, function (item) { return item[0]; });
dollarFirstChars = _.filter(firstChars, function (c) { return c === '$'; });
if (dollarFirstChars.length !== 0 && dollarFirstChars.length !== firstChars.length) {
throw new Error("You cannot mix operators and normal fields");
}
// queryValue is an object of this form: { $comparisonOperator1: value1, ... }
if (dollarFirstChars.length > 0) {
for (i = 0; i < keys.length; i += 1) {
if (!comparisonFunctions[keys[i]]) { throw new Error("Unknown comparison function " + keys[i]); }
if (!comparisonFunctions[keys[i]](objValue, queryValue[keys[i]])) { return false; }
}
return true;
}
}
// Using regular expressions with basic querying
if (util.isRegExp(queryValue)) { return comparisonFunctions.$regex(objValue, queryValue); }
// queryValue is either a native value or a normal object
// Basic matching is possible
if (!areThingsEqual(objValue, queryValue)) { return false; }
return true;
}
// Interface
module.exports.serialize = serialize;
module.exports.deserialize = deserialize;
module.exports.deepCopy = deepCopy;
module.exports.checkObject = checkObject;
module.exports.isPrimitiveType = isPrimitiveType;
module.exports.modify = modify;
module.exports.getDotValue = getDotValue;
module.exports.match = match;
module.exports.areThingsEqual = areThingsEqual;
module.exports.compareThings = compareThings;
},{"underscore":19,"util":3}],11:[function(require,module,exports){
var process=require("__browserify_process");/**
* Handle every persistence-related task
* The interface Datastore expects to be implemented is
* * Persistence.loadDatabase(callback) and callback has signature err
* * Persistence.persistNewState(newDocs, callback) where newDocs is an array of documents and callback has signature err
*/
var storage = require('./storage')
, path = require('path')
, model = require('./model')
, async = require('async')
, customUtils = require('./customUtils')
, Index = require('./indexes')
;
/**
* Create a new Persistence object for database options.db
* @param {Datastore} options.db
* @param {Boolean} options.nodeWebkitAppName Optional, specify the name of your NW app if you want options.filename to be relative to the directory where
* Node Webkit stores application data such as cookies and local storage (the best place to store data in my opinion)
*/
function Persistence (options) {
var i, j, randomString;
this.db = options.db;
this.inMemoryOnly = this.db.inMemoryOnly;
this.filename = this.db.filename;
this.corruptAlertThreshold = options.corruptAlertThreshold !== undefined ? options.corruptAlertThreshold : 0.1;
if (!this.inMemoryOnly && this.filename && this.filename.charAt(this.filename.length - 1) === '~') {
throw new Error("The datafile name can't end with a ~, which is reserved for crash safe backup files");
}
// After serialization and before deserialization hooks with some basic sanity checks
if (options.afterSerialization && !options.beforeDeserialization) {
throw new Error("Serialization hook defined but deserialization hook undefined, cautiously refusing to start NeDB to prevent dataloss");
}
if (!options.afterSerialization && options.beforeDeserialization) {
throw new Error("Serialization hook undefined but deserialization hook defined, cautiously refusing to start NeDB to prevent dataloss");
}
this.afterSerialization = options.afterSerialization || function (s) { return s; };
this.beforeDeserialization = options.beforeDeserialization || function (s) { return s; };
for (i = 1; i < 30; i += 1) {
for (j = 0; j < 10; j += 1) {
randomString = customUtils.uid(i);
if (this.beforeDeserialization(this.afterSerialization(randomString)) !== randomString) {
throw new Error("beforeDeserialization is not the reverse of afterSerialization, cautiously refusing to start NeDB to prevent dataloss");
}
}
}
// For NW apps, store data in the same directory where NW stores application data
if (this.filename && options.nodeWebkitAppName) {
console.log("==================================================================");
console.log("WARNING: The nodeWebkitAppName option is deprecated");
console.log("To get the path to the directory where Node Webkit stores the data");
console.log("for your app, use the internal nw.gui module like this");
console.log("require('nw.gui').App.dataPath");
console.log("See https://github.com/rogerwang/node-webkit/issues/500");
console.log("==================================================================");
this.filename = Persistence.getNWAppFilename(options.nodeWebkitAppName, this.filename);
}
};
/**
* Check if a directory exists and create it on the fly if it is not the case
* cb is optional, signature: err
*/
Persistence.ensureDirectoryExists = function (dir, cb) {
var callback = cb || function () {}
;
storage.mkdirp(dir, function (err) { return callback(err); });
};
/**
* Return the path the datafile if the given filename is relative to the directory where Node Webkit stores
* data for this application. Probably the best place to store data
*/
Persistence.getNWAppFilename = function (appName, relativeFilename) {
var home;
switch (process.platform) {
case 'win32':
case 'win64':
home = process.env.LOCALAPPDATA || process.env.APPDATA;
if (!home) { throw new Error("Couldn't find the base application data folder"); }
home = path.join(home, appName);
break;
case 'darwin':
home = process.env.HOME;
if (!home) { throw new Error("Couldn't find the base application data directory"); }
home = path.join(home, 'Library', 'Application Support', appName);
break;
case 'linux':
home = process.env.HOME;
if (!home) { throw new Error("Couldn't find the base application data directory"); }
home = path.join(home, '.config', appName);
break;
default:
throw new Error("Can't use the Node Webkit relative path for platform " + process.platform);
break;
}
return path.join(home, 'nedb-data', relativeFilename);
}
/**
* Persist cached database
* This serves as a compaction function since the cache always contains only the number of documents in the collection
* while the data file is append-only so it may grow larger
* @param {Function} cb Optional callback, signature: err
*/
Persistence.prototype.persistCachedDatabase = function (cb) {
var callback = cb || function () {}
, toPersist = ''
, self = this
;
if (this.inMemoryOnly) { return callback(null); }
this.db.getAllData().forEach(function (doc) {
toPersist += self.afterSerialization(model.serialize(doc)) + '\n';
});
Object.keys(this.db.indexes).forEach(function (fieldName) {
if (fieldName != "_id") { // The special _id index is managed by datastore.js, the others need to be persisted
toPersist += self.afterSerialization(model.serialize({ $$indexCreated: { fieldName: fieldName, unique: self.db.indexes[fieldName].unique, sparse: self.db.indexes[fieldName].sparse }})) + '\n';
}
});
storage.crashSafeWriteFile(this.filename, toPersist, function (err) {
if (err) { return callback(err); }
self.db.emit('compaction.done');
return callback(null);
});
};
/**
* Queue a rewrite of the datafile
*/
Persistence.prototype.compactDatafile = function () {
this.db.executor.push({ this: this, fn: this.persistCachedDatabase, arguments: [] });
};
/**
* Set automatic compaction every interval ms
* @param {Number} interval in milliseconds, with an enforced minimum of 5 seconds
*/
Persistence.prototype.setAutocompactionInterval = function (interval) {
var self = this
, minInterval = 5000
, realInterval = Math.max(interval || 0, minInterval)
;
this.stopAutocompaction();
this.autocompactionIntervalId = setInterval(function () {
self.compactDatafile();
}, realInterval);
};
/**
* Stop autocompaction (do nothing if autocompaction was not running)
*/
Persistence.prototype.stopAutocompaction = function () {
if (this.autocompactionIntervalId) { clearInterval(this.autocompactionIntervalId); }
};
/**
* Persist new state for the given newDocs (can be insertion, update or removal)
* Use an append-only format
* @param {Array} newDocs Can be empty if no doc was updated/removed
* @param {Function} cb Optional, signature: err
*/
Persistence.prototype.persistNewState = function (newDocs, cb) {
var self = this
, toPersist = ''
, callback = cb || function () {}
;
// In-memory only datastore
if (self.inMemoryOnly) { return callback(null); }
newDocs.forEach(function (doc) {
toPersist += self.afterSerialization(model.serialize(doc)) + '\n';
});
if (toPersist.length === 0) { return callback(null); }
storage.appendFile(self.filename, toPersist, 'utf8', function (err) {
return callback(err);
});
};
/**
* From a database's raw data, return the corresponding
* machine understandable collection
*/
Persistence.prototype.treatRawData = function (rawData) {
var data = rawData.split('\n')
, dataById = {}
, tdata = []
, i
, indexes = {}
, corruptItems = -1 // Last line of every data file is usually blank so not really corrupt
;
for (i = 0; i < data.length; i += 1) {
var doc;
try {
doc = model.deserialize(this.beforeDeserialization(data[i]));
if (doc._id) {
if (doc.$$deleted === true) {
delete dataById[doc._id];
} else {
dataById[doc._id] = doc;
}
} else if (doc.$$indexCreated && doc.$$indexCreated.fieldName != undefined) {
indexes[doc.$$indexCreated.fieldName] = doc.$$indexCreated;
} else if (typeof doc.$$indexRemoved === "string") {
delete indexes[doc.$$indexRemoved];
}
} catch (e) {
corruptItems += 1;
}
}
// A bit lenient on corruption
if (data.length > 0 && corruptItems / data.length > this.corruptAlertThreshold) {
throw new Error("More than " + Math.floor(100 * this.corruptAlertThreshold) + "% of the data file is corrupt, the wrong beforeDeserialization hook may be used. Cautiously refusing to start NeDB to prevent dataloss");
}
Object.keys(dataById).forEach(function (k) {
tdata.push(dataById[k]);
});
return { data: tdata, indexes: indexes };
};
/**
* Load the database
* 1) Create all indexes
* 2) Insert all data
* 3) Compact the database
* This means pulling data out of the data file or creating it if it doesn't exist
* Also, all data is persisted right away, which has the effect of compacting the database file
* This operation is very quick at startup for a big collection (60ms for ~10k docs)
* @param {Function} cb Optional callback, signature: err
*/
Persistence.prototype.loadDatabase = function (cb) {
var callback = cb || function () {}
, self = this
;
self.db.resetIndexes();
// In-memory only datastore
if (self.inMemoryOnly) { return callback(null); }
async.waterfall([
function (cb) {
Persistence.ensureDirectoryExists(path.dirname(self.filename), function (err) {
storage.ensureDatafileIntegrity(self.filename, function (err) {
storage.readFile(self.filename, 'utf8', function (err, rawData) {
if (err) { return cb(err); }
try {
var treatedData = self.treatRawData(rawData);
} catch (e) {
return cb(e);
}
// Recreate all indexes in the datafile
Object.keys(treatedData.indexes).forEach(function (key) {
self.db.indexes[key] = new Index(treatedData.indexes[key]);
});
// Fill cached database (i.e. all indexes) with data
try {
self.db.resetIndexes(treatedData.data);
} catch (e) {
self.db.resetIndexes(); // Rollback any index which didn't fail
return cb(e);
}
self.db.persistence.persistCachedDatabase(cb);
});
});
});
}
], function (err) {
if (err) { return callback(err); }
self.db.executor.processBuffer();
return callback(null);
});
};
// Interface
module.exports = Persistence;
},{"./customUtils":6,"./indexes":9,"./model":10,"./storage":12,"__browserify_process":4,"async":13,"path":2}],12:[function(require,module,exports){
/**
* Way data is stored for this database
* For a Node.js/Node Webkit database it's the file system
* For a browser-side database it's localforage, which uses the best backend available (IndexedDB then WebSQL then localStorage)
*
* This version is the browser version
*/
var localforage = require('localforage')
// Configure localforage to display NeDB name for now. Would be a good idea to let user use his own app name
localforage.config({
name: 'NeDB'
, storeName: 'nedbdata'
});
function exists (filename, callback) {
localforage.getItem(filename, function (err, value) {
if (value !== null) { // Even if value is undefined, localforage returns null
return callback(true);
} else {
return callback(false);
}
});
}
function rename (filename, newFilename, callback) {
localforage.getItem(filename, function (err, value) {
if (value === null) {
localforage.removeItem(newFilename, function () { return callback(); });
} else {
localforage.setItem(newFilename, value, function () {
localforage.removeItem(filename, function () { return callback(); });
});
}
});
}
function writeFile (filename, contents, options, callback) {
// Options do not matter in browser setup
if (typeof options === 'function') { callback = options; }
localforage.setItem(filename, contents, function () { return callback(); });
}
function appendFile (filename, toAppend, options, callback) {
// Options do not matter in browser setup
if (typeof options === 'function') { callback = options; }
localforage.getItem(filename, function (err, contents) {
contents = contents || '';
contents += toAppend;
localforage.setItem(filename, contents, function () { return callback(); });
});
}
function readFile (filename, options, callback) {
// Options do not matter in browser setup
if (typeof options === 'function') { callback = options; }
localforage.getItem(filename, function (err, contents) { return callback(null, contents || ''); });
}
function unlink (filename, callback) {
localforage.removeItem(filename, function () { return callback(); });
}
// Nothing to do, no directories will be used on the browser
function mkdirp (dir, callback) {
return callback();
}
// Nothing to do, no data corruption possible in the brower
function ensureDatafileIntegrity (filename, callback) {
return callback(null);
}
// Interface
module.exports.exists = exists;
module.exports.rename = rename;
module.exports.writeFile = writeFile;
module.exports.crashSafeWriteFile = writeFile; // No need for a crash safe function in the browser
module.exports.appendFile = appendFile;
module.exports.readFile = readFile;
module.exports.unlink = unlink;
module.exports.mkdirp = mkdirp;
module.exports.ensureDatafileIntegrity = ensureDatafileIntegrity;
},{"localforage":18}],13:[function(require,module,exports){
var process=require("__browserify_process");/*global setImmediate: false, setTimeout: false, console: false */
(function () {
var async = {};
// global on the server, window in the browser
var root, previous_async;
root = this;
if (root != null) {
previous_async = root.async;
}
async.noConflict = function () {
root.async = previous_async;
return async;
};
function only_once(fn) {
var called = false;
return function() {
if (called) throw new Error("Callback was already called.");
called = true;
fn.apply(root, arguments);
}
}
//// cross-browser compatiblity functions ////
var _each = function (arr, iterator) {
if (arr.forEach) {
return arr.forEach(iterator);
}
for (var i = 0; i < arr.length; i += 1) {
iterator(arr[i], i, arr);
}
};
var _map = function (arr, iterator) {
if (arr.map) {
return arr.map(iterator);
}
var results = [];
_each(arr, function (x, i, a) {
results.push(iterator(x, i, a));
});
return results;
};
var _reduce = function (arr, iterator, memo) {
if (arr.reduce) {
return arr.reduce(iterator, memo);
}
_each(arr, function (x, i, a) {
memo = iterator(memo, x, i, a);
});
return memo;
};
var _keys = function (obj) {
if (Object.keys) {
return Object.keys(obj);
}
var keys = [];
for (var k in obj) {
if (obj.hasOwnProperty(k)) {
keys.push(k);
}
}
return keys;
};
//// exported async module functions ////
//// nextTick implementation with browser-compatible fallback ////
if (typeof process === 'undefined' || !(process.nextTick)) {
if (typeof setImmediate === 'function') {
async.nextTick = function (fn) {
// not a direct alias for IE10 compatibility
setImmediate(fn);
};
async.setImmediate = async.nextTick;
}
else {
async.nextTick = function (fn) {
setTimeout(fn, 0);
};
async.setImmediate = async.nextTick;
}
}
else {
async.nextTick = process.nextTick;
if (typeof setImmediate !== 'undefined') {
async.setImmediate = function (fn) {
// not a direct alias for IE10 compatibility
setImmediate(fn);
};
}
else {
async.setImmediate = async.nextTick;
}
}
async.each = function (arr, iterator, callback) {
callback = callback || function () {};
if (!arr.length) {
return callback();
}
var completed = 0;
_each(arr, function (x) {
iterator(x, only_once(function (err) {
if (err) {
callback(err);
callback = function () {};
}
else {
completed += 1;
if (completed >= arr.length) {
callback(null);
}
}
}));
});
};
async.forEach = async.each;
async.eachSeries = function (arr, iterator, callback) {
callback = callback || function () {};
if (!arr.length) {
return callback();
}
var completed = 0;
var iterate = function () {
iterator(arr[completed], function (err) {
if (err) {
callback(err);
callback = function () {};
}
else {
completed += 1;
if (completed >= arr.length) {
callback(null);
}
else {
iterate();
}
}
});
};
iterate();
};
async.forEachSeries = async.eachSeries;
async.eachLimit = function (arr, limit, iterator, callback) {
var fn = _eachLimit(limit);
fn.apply(null, [arr, iterator, callback]);
};
async.forEachLimit = async.eachLimit;
var _eachLimit = function (limit) {
return function (arr, iterator, callback) {
callback = callback || function () {};
if (!arr.length || limit <= 0) {
return callback();
}
var completed = 0;
var started = 0;
var running = 0;
(function replenish () {
if (completed >= arr.length) {
return callback();
}
while (running < limit && started < arr.length) {
started += 1;
running += 1;
iterator(arr[started - 1], function (err) {
if (err) {
callback(err);
callback = function () {};
}
else {
completed += 1;
running -= 1;
if (completed >= arr.length) {
callback();
}
else {
replenish();
}
}
});
}
})();
};
};
var doParallel = function (fn) {
return function () {
var args = Array.prototype.slice.call(arguments);
return fn.apply(null, [async.each].concat(args));
};
};
var doParallelLimit = function(limit, fn) {
return function () {
var args = Array.prototype.slice.call(arguments);
return fn.apply(null, [_eachLimit(limit)].concat(args));
};
};
var doSeries = function (fn) {
return function () {
var args = Array.prototype.slice.call(arguments);
return fn.apply(null, [async.eachSeries].concat(args));
};
};
var _asyncMap = function (eachfn, arr, iterator, callback) {
var results = [];
arr = _map(arr, function (x, i) {
return {index: i, value: x};
});
eachfn(arr, function (x, callback) {
iterator(x.value, function (err, v) {
results[x.index] = v;
callback(err);
});
}, function (err) {
callback(err, results);
});
};
async.map = doParallel(_asyncMap);
async.mapSeries = doSeries(_asyncMap);
async.mapLimit = function (arr, limit, iterator, callback) {
return _mapLimit(limit)(arr, iterator, callback);
};
var _mapLimit = function(limit) {
return doParallelLimit(limit, _asyncMap);
};
// reduce only has a series version, as doing reduce in parallel won't
// work in many situations.
async.reduce = function (arr, memo, iterator, callback) {
async.eachSeries(arr, function (x, callback) {
iterator(memo, x, function (err, v) {
memo = v;
callback(err);
});
}, function (err) {
callback(err, memo);
});
};
// inject alias
async.inject = async.reduce;
// foldl alias
async.foldl = async.reduce;
async.reduceRight = function (arr, memo, iterator, callback) {
var reversed = _map(arr, function (x) {
return x;
}).reverse();
async.reduce(reversed, memo, iterator, callback);
};
// foldr alias
async.foldr = async.reduceRight;
var _filter = function (eachfn, arr, iterator, callback) {
var results = [];
arr = _map(arr, function (x, i) {
return {index: i, value: x};
});
eachfn(arr, function (x, callback) {
iterator(x.value, function (v) {
if (v) {
results.push(x);
}
callback();
});
}, function (err) {
callback(_map(results.sort(function (a, b) {
return a.index - b.index;
}), function (x) {
return x.value;
}));
});
};
async.filter = doParallel(_filter);
async.filterSeries = doSeries(_filter);
// select alias
async.select = async.filter;
async.selectSeries = async.filterSeries;
var _reject = function (eachfn, arr, iterator, callback) {
var results = [];
arr = _map(arr, function (x, i) {
return {index: i, value: x};
});
eachfn(arr, function (x, callback) {
iterator(x.value, function (v) {
if (!v) {
results.push(x);
}
callback();
});
}, function (err) {
callback(_map(results.sort(function (a, b) {
return a.index - b.index;
}), function (x) {
return x.value;
}));
});
};
async.reject = doParallel(_reject);
async.rejectSeries = doSeries(_reject);
var _detect = function (eachfn, arr, iterator, main_callback) {
eachfn(arr, function (x, callback) {
iterator(x, function (result) {
if (result) {
main_callback(x);
main_callback = function () {};
}
else {
callback();
}
});
}, function (err) {
main_callback();
});
};
async.detect = doParallel(_detect);
async.detectSeries = doSeries(_detect);
async.some = function (arr, iterator, main_callback) {
async.each(arr, function (x, callback) {
iterator(x, function (v) {
if (v) {
main_callback(true);
main_callback = function () {};
}
callback();
});
}, function (err) {
main_callback(false);
});
};
// any alias
async.any = async.some;
async.every = function (arr, iterator, main_callback) {
async.each(arr, function (x, callback) {
iterator(x, function (v) {
if (!v) {
main_callback(false);
main_callback = function () {};
}
callback();
});
}, function (err) {
main_callback(true);
});
};
// all alias
async.all = async.every;
async.sortBy = function (arr, iterator, callback) {
async.map(arr, function (x, callback) {
iterator(x, function (err, criteria) {
if (err) {
callback(err);
}
else {
callback(null, {value: x, criteria: criteria});
}
});
}, function (err, results) {
if (err) {
return callback(err);
}
else {
var fn = function (left, right) {
var a = left.criteria, b = right.criteria;
return a < b ? -1 : a > b ? 1 : 0;
};
callback(null, _map(results.sort(fn), function (x) {
return x.value;
}));
}
});
};
async.auto = function (tasks, callback) {
callback = callback || function () {};
var keys = _keys(tasks);
if (!keys.length) {
return callback(null);
}
var results = {};
var listeners = [];
var addListener = function (fn) {
listeners.unshift(fn);
};
var removeListener = function (fn) {
for (var i = 0; i < listeners.length; i += 1) {
if (listeners[i] === fn) {
listeners.splice(i, 1);
return;
}
}
};
var taskComplete = function () {
_each(listeners.slice(0), function (fn) {
fn();
});
};
addListener(function () {
if (_keys(results).length === keys.length) {
callback(null, results);
callback = function () {};
}
});
_each(keys, function (k) {
var task = (tasks[k] instanceof Function) ? [tasks[k]]: tasks[k];
var taskCallback = function (err) {
var args = Array.prototype.slice.call(arguments, 1);
if (args.length <= 1) {
args = args[0];
}
if (err) {
var safeResults = {};
_each(_keys(results), function(rkey) {
safeResults[rkey] = results[rkey];
});
safeResults[k] = args;
callback(err, safeResults);
// stop subsequent errors hitting callback multiple times
callback = function () {};
}
else {
results[k] = args;
async.setImmediate(taskComplete);
}
};
var requires = task.slice(0, Math.abs(task.length - 1)) || [];
var ready = function () {
return _reduce(requires, function (a, x) {
return (a && results.hasOwnProperty(x));
}, true) && !results.hasOwnProperty(k);
};
if (ready()) {
task[task.length - 1](taskCallback, results);
}
else {
var listener = function () {
if (ready()) {
removeListener(listener);
task[task.length - 1](taskCallback, results);
}
};
addListener(listener);
}
});
};
async.waterfall = function (tasks, callback) {
callback = callback || function () {};
if (tasks.constructor !== Array) {
var err = new Error('First argument to waterfall must be an array of functions');
return callback(err);
}
if (!tasks.length) {
return callback();
}
var wrapIterator = function (iterator) {
return function (err) {
if (err) {
callback.apply(null, arguments);
callback = function () {};
}
else {
var args = Array.prototype.slice.call(arguments, 1);
var next = iterator.next();
if (next) {
args.push(wrapIterator(next));
}
else {
args.push(callback);
}
async.setImmediate(function () {
iterator.apply(null, args);
});
}
};
};
wrapIterator(async.iterator(tasks))();
};
var _parallel = function(eachfn, tasks, callback) {
callback = callback || function () {};
if (tasks.constructor === Array) {
eachfn.map(tasks, function (fn, callback) {
if (fn) {
fn(function (err) {
var args = Array.prototype.slice.call(arguments, 1);
if (args.length <= 1) {
args = args[0];
}
callback.call(null, err, args);
});
}
}, callback);
}
else {
var results = {};
eachfn.each(_keys(tasks), function (k, callback) {
tasks[k](function (err) {
var args = Array.prototype.slice.call(arguments, 1);
if (args.length <= 1) {
args = args[0];
}
results[k] = args;
callback(err);
});
}, function (err) {
callback(err, results);
});
}
};
async.parallel = function (tasks, callback) {
_parallel({ map: async.map, each: async.each }, tasks, callback);
};
async.parallelLimit = function(tasks, limit, callback) {
_parallel({ map: _mapLimit(limit), each: _eachLimit(limit) }, tasks, callback);
};
async.series = function (tasks, callback) {
callback = callback || function () {};
if (tasks.constructor === Array) {
async.mapSeries(tasks, function (fn, callback) {
if (fn) {
fn(function (err) {
var args = Array.prototype.slice.call(arguments, 1);
if (args.length <= 1) {
args = args[0];
}
callback.call(null, err, args);
});
}
}, callback);
}
else {
var results = {};
async.eachSeries(_keys(tasks), function (k, callback) {
tasks[k](function (err) {
var args = Array.prototype.slice.call(arguments, 1);
if (args.length <= 1) {
args = args[0];
}
results[k] = args;
callback(err);
});
}, function (err) {
callback(err, results);
});
}
};
async.iterator = function (tasks) {
var makeCallback = function (index) {
var fn = function () {
if (tasks.length) {
tasks[index].apply(null, arguments);
}
return fn.next();
};
fn.next = function () {
return (index < tasks.length - 1) ? makeCallback(index + 1): null;
};
return fn;
};
return makeCallback(0);
};
async.apply = function (fn) {
var args = Array.prototype.slice.call(arguments, 1);
return function () {
return fn.apply(
null, args.concat(Array.prototype.slice.call(arguments))
);
};
};
var _concat = function (eachfn, arr, fn, callback) {
var r = [];
eachfn(arr, function (x, cb) {
fn(x, function (err, y) {
r = r.concat(y || []);
cb(err);
});
}, function (err) {
callback(err, r);
});
};
async.concat = doParallel(_concat);
async.concatSeries = doSeries(_concat);
async.whilst = function (test, iterator, callback) {
if (test()) {
iterator(function (err) {
if (err) {
return callback(err);
}
async.whilst(test, iterator, callback);
});
}
else {
callback();
}
};
async.doWhilst = function (iterator, test, callback) {
iterator(function (err) {
if (err) {
return callback(err);
}
if (test()) {
async.doWhilst(iterator, test, callback);
}
else {
callback();
}
});
};
async.until = function (test, iterator, callback) {
if (!test()) {
iterator(function (err) {
if (err) {
return callback(err);
}
async.until(test, iterator, callback);
});
}
else {
callback();
}
};
async.doUntil = function (iterator, test, callback) {
iterator(function (err) {
if (err) {
return callback(err);
}
if (!test()) {
async.doUntil(iterator, test, callback);
}
else {
callback();
}
});
};
async.queue = function (worker, concurrency) {
if (concurrency === undefined) {
concurrency = 1;
}
function _insert(q, data, pos, callback) {
if(data.constructor !== Array) {
data = [data];
}
_each(data, function(task) {
var item = {
data: task,
callback: typeof callback === 'function' ? callback : null
};
if (pos) {
q.tasks.unshift(item);
} else {
q.tasks.push(item);
}
if (q.saturated && q.tasks.length === concurrency) {
q.saturated();
}
async.setImmediate(q.process);
});
}
var workers = 0;
var q = {
tasks: [],
concurrency: concurrency,
saturated: null,
empty: null,
drain: null,
push: function (data, callback) {
_insert(q, data, false, callback);
},
unshift: function (data, callback) {
_insert(q, data, true, callback);
},
process: function () {
if (workers < q.concurrency && q.tasks.length) {
var task = q.tasks.shift();
if (q.empty && q.tasks.length === 0) {
q.empty();
}
workers += 1;
var next = function () {
workers -= 1;
if (task.callback) {
task.callback.apply(task, arguments);
}
if (q.drain && q.tasks.length + workers === 0) {
q.drain();
}
q.process();
};
var cb = only_once(next);
worker(task.data, cb);
}
},
length: function () {
return q.tasks.length;
},
running: function () {
return workers;
}
};
return q;
};
async.cargo = function (worker, payload) {
var working = false,
tasks = [];
var cargo = {
tasks: tasks,
payload: payload,
saturated: null,
empty: null,
drain: null,
push: function (data, callback) {
if(data.constructor !== Array) {
data = [data];
}
_each(data, function(task) {
tasks.push({
data: task,
callback: typeof callback === 'function' ? callback : null
});
if (cargo.saturated && tasks.length === payload) {
cargo.saturated();
}
});
async.setImmediate(cargo.process);
},
process: function process() {
if (working) return;
if (tasks.length === 0) {
if(cargo.drain) cargo.drain();
return;
}
var ts = typeof payload === 'number'
? tasks.splice(0, payload)
: tasks.splice(0);
var ds = _map(ts, function (task) {
return task.data;
});
if(cargo.empty) cargo.empty();
working = true;
worker(ds, function () {
working = false;
var args = arguments;
_each(ts, function (data) {
if (data.callback) {
data.callback.apply(null, args);
}
});
process();
});
},
length: function () {
return tasks.length;
},
running: function () {
return working;
}
};
return cargo;
};
var _console_fn = function (name) {
return function (fn) {
var args = Array.prototype.slice.call(arguments, 1);
fn.apply(null, args.concat([function (err) {
var args = Array.prototype.slice.call(arguments, 1);
if (typeof console !== 'undefined') {
if (err) {
if (console.error) {
console.error(err);
}
}
else if (console[name]) {
_each(args, function (x) {
console[name](x);
});
}
}
}]));
};
};
async.
gitextract_fo51vwzk/
├── .gitignore
├── LICENSE
├── README.md
├── benchmarks/
│ ├── commonUtilities.js
│ ├── ensureIndex.js
│ ├── find.js
│ ├── findOne.js
│ ├── findWithIn.js
│ ├── insert.js
│ ├── loadDatabase.js
│ ├── remove.js
│ └── update.js
├── bower.json
├── browser-version/
│ ├── browser-specific/
│ │ └── lib/
│ │ ├── customUtils.js
│ │ └── storage.js
│ ├── build.js
│ ├── out/
│ │ └── nedb.js
│ ├── package.json
│ └── test/
│ ├── async.js
│ ├── chai.js
│ ├── index.html
│ ├── localforage.js
│ ├── mocha.css
│ ├── mocha.js
│ ├── nedb-browser.js
│ ├── playground.html
│ ├── testLoad.html
│ ├── testLoad.js
│ ├── testPersistence.html
│ ├── testPersistence.js
│ ├── testPersistence2.html
│ └── testPersistence2.js
├── index.js
├── lib/
│ ├── cursor.js
│ ├── customUtils.js
│ ├── datastore.js
│ ├── executor.js
│ ├── indexes.js
│ ├── model.js
│ ├── persistence.js
│ └── storage.js
├── package.json
├── test/
│ ├── cursor.test.js
│ ├── customUtil.test.js
│ ├── db.test.js
│ ├── executor.test.js
│ ├── indexes.test.js
│ ├── mocha.opts
│ ├── model.test.js
│ └── persistence.test.js
└── test_lac/
├── loadAndCrash.test.js
├── openFds.test.js
├── openFdsLaunch.sh
├── openFdsTestFile
└── openFdsTestFile2
SYMBOL INDEX (454 symbols across 24 files)
FILE: benchmarks/commonUtilities.js
function getRandomArray (line 69) | function getRandomArray (n) {
function runFrom (line 98) | function runFrom(i) {
function runFrom (line 127) | function runFrom(i) {
function runFrom (line 166) | function runFrom(i) {
function runFrom (line 194) | function runFrom(i) {
function runFrom (line 223) | function runFrom(i) {
function runFrom (line 254) | function runFrom(i) {
function runFrom (line 290) | function runFrom(i) {
FILE: browser-version/browser-specific/lib/customUtils.js
function randomBytes (line 10) | function randomBytes (size) {
function byteArrayToBase64 (line 27) | function byteArrayToBase64 (uint8) {
function uid (line 72) | function uid (len) {
FILE: browser-version/browser-specific/lib/storage.js
function exists (line 18) | function exists (filename, callback) {
function rename (line 29) | function rename (filename, newFilename, callback) {
function writeFile (line 42) | function writeFile (filename, contents, options, callback) {
function appendFile (line 49) | function appendFile (filename, toAppend, options, callback) {
function readFile (line 61) | function readFile (filename, options, callback) {
function unlink (line 68) | function unlink (filename, callback) {
function mkdirp (line 74) | function mkdirp (dir, callback) {
function ensureDatafileIntegrity (line 80) | function ensureDatafileIntegrity (filename, callback) {
FILE: browser-version/build.js
function ensureDirExists (line 13) | function ensureDirExists (name) {
FILE: browser-version/out/nedb.js
function i (line 2) | function i(n,s){if(!t[n]){if(!e[n]){var o=typeof require=="function"&&re...
function indexOf (line 12) | function indexOf (xs, x) {
function filter (line 199) | function filter (xs, fn) {
function normalizeArray (line 211) | function normalizeArray(parts, allowAboveRoot) {
function trim (line 338) | function trim(arr) {
function format (line 431) | function format(value, recurseTimes) {
function isArray (line 606) | function isArray(ar) {
function isRegExp (line 612) | function isRegExp(re) {
function isDate (line 617) | function isDate(d) {
function pad (line 621) | function pad(n) {
function timestamp (line 629) | function timestamp() {
function Cursor (line 795) | function Cursor (db, query, execFn) {
function callback (line 903) | function callback (error, res) {
function randomBytes (line 995) | function randomBytes (size) {
function byteArrayToBase64 (line 1012) | function byteArrayToBase64 (uint8) {
function uid (line 1057) | function uid (len) {
function Datastore (line 1094) | function Datastore (options) {
function Executor (line 1780) | function Executor () {
function checkValueEquality (line 1862) | function checkValueEquality (a, b) {
function projectForUnique (line 1869) | function projectForUnique (elt) {
function Index (line 1888) | function Index (options) {
function checkKey (line 2174) | function checkKey (k, v) {
function checkObject (line 2193) | function checkObject (obj) {
function serialize (line 2217) | function serialize (obj) {
function deserialize (line 2241) | function deserialize (rawData) {
function deepCopy (line 2257) | function deepCopy (obj, strictKeys) {
function isPrimitiveType (line 2292) | function isPrimitiveType (obj) {
function compareNSB (line 2307) | function compareNSB (a, b) {
function compareArrays (line 2313) | function compareArrays (a, b) {
function compareThings (line 2337) | function compareThings (a, b, _compareStrings) {
function createModifierFunction (line 2560) | function createModifierFunction (modifier) {
function modify (line 2585) | function modify (obj, updateQuery) {
function getDotValue (line 2641) | function getDotValue (obj, field) {
function areThingsEqual (line 2676) | function areThingsEqual (a, b) {
function areComparable (line 2711) | function areComparable (a, b) {
function match (line 2882) | function match (obj, query) {
function matchQueryPart (line 2914) | function matchQueryPart (obj, queryKey, queryValue, treatObjAsValue) {
function Persistence (line 3008) | function Persistence (options) {
function exists (line 3319) | function exists (filename, callback) {
function rename (line 3330) | function rename (filename, newFilename, callback) {
function writeFile (line 3343) | function writeFile (filename, contents, options, callback) {
function appendFile (line 3350) | function appendFile (filename, toAppend, options, callback) {
function readFile (line 3362) | function readFile (filename, options, callback) {
function unlink (line 3369) | function unlink (filename, callback) {
function mkdirp (line 3375) | function mkdirp (dir, callback) {
function ensureDatafileIntegrity (line 3381) | function ensureDatafileIntegrity (filename, callback) {
function only_once (line 3417) | function only_once(fn) {
function _insert (line 4075) | function _insert(q, data, pos, callback) {
function next (line 4329) | function next(err) {
function AVLTree (line 4380) | function AVLTree (options) {
function _AVLTree (line 4393) | function _AVLTree (options) {
function BinarySearchTree (line 4834) | function BinarySearchTree (options) {
function append (line 5151) | function append (array, toAppend) {
function getRandomArray (line 5368) | function getRandomArray (n) {
function defaultCompareKeysFunction (line 5386) | function defaultCompareKeysFunction (a, b) {
function defaultCheckValueEquality (line 5402) | function defaultCheckValueEquality (a, b) {
function resolve (line 5451) | function resolve(child) {
function all (line 5523) | function all(promises) {
function useNextTick (line 5575) | function useNextTick() {
function useMutationObserver (line 5581) | function useMutationObserver() {
function useSetTimeout (line 5592) | function useSetTimeout() {
function flush (line 5599) | function flush() {
function asap (line 5619) | function asap(callback, arg) {
function configure (line 5639) | function configure(name, value) {
function polyfill (line 5658) | function polyfill() {
function Promise (line 5711) | function Promise(resolver) {
function invokeResolver (line 5725) | function invokeResolver(resolver, promise) {
function invokeCallback (line 5741) | function invokeCallback(settled, promise, callback, detail) {
function subscribe (line 5776) | function subscribe(parent, child, onFulfillment, onRejection) {
function publish (line 5785) | function publish(promise, settled) {
function handleThenable (line 5832) | function handleThenable(promise, value) {
function resolve (line 5873) | function resolve(promise, value) {
function fulfill (line 5881) | function fulfill(promise, value) {
function reject (line 5889) | function reject(promise, reason) {
function publishFulfillment (line 5897) | function publishFulfillment(promise) {
function publishRejection (line 5901) | function publishRejection(promise) {
function race (line 5976) | function race(promises) {
function reject (line 6040) | function reject(reason) {
function resolve (line 6055) | function resolve(value) {
function objectOrFunction (line 6074) | function objectOrFunction(x) {
function isFunction (line 6078) | function isFunction(x) {
function isArray (line 6082) | function isArray(x) {
function __webpack_require__ (line 6112) | function __webpack_require__(moduleId) {
function _classCallCheck (line 6157) | function _classCallCheck(instance, Constructor) { if (!(instance instanc...
function callWhenReady (line 6240) | function callWhenReady(localForageInstance, libraryMethod) {
function extend (line 6249) | function extend() {
function isLibraryDriver (line 6269) | function isLibraryDriver(driverName) {
function LocalForage (line 6280) | function LocalForage(options) {
function setDriverToConfig (line 6461) | function setDriverToConfig() {
function initDriver (line 6465) | function initDriver(supportedDrivers) {
function _createBlob (line 6595) | function _createBlob(parts, properties) {
function _binStringToArrayBuffer (line 6618) | function _binStringToArrayBuffer(bin) {
function _blobAjax (line 6631) | function _blobAjax(url) {
function _checkBlobSupportWithoutCaching (line 6669) | function _checkBlobSupportWithoutCaching(idb) {
function _checkBlobSupport (line 6699) | function _checkBlobSupport(idb) {
function _encodeBlob (line 6710) | function _encodeBlob(blob) {
function _decodeBlob (line 6727) | function _decodeBlob(encodedBlob) {
function _isEncodedBlob (line 6733) | function _isEncodedBlob(value) {
function _initStorage (line 6739) | function _initStorage(options) {
function _getOriginalConnection (line 6822) | function _getOriginalConnection(dbInfo) {
function _getUpgradedConnection (line 6826) | function _getUpgradedConnection(dbInfo) {
function _getConnection (line 6830) | function _getConnection(dbInfo, upgradeNeeded) {
function _isUpgradeNeeded (line 6877) | function _isUpgradeNeeded(dbInfo, defaultVersion) {
function getItem (line 6913) | function getItem(key, callback) {
function iterate (line 6950) | function iterate(iterator, callback) {
function setItem (line 6992) | function setItem(key, value, callback) {
function removeItem (line 7048) | function removeItem(key, callback) {
function clear (line 7090) | function clear(callback) {
function length (line 7115) | function length(callback) {
function key (line 7138) | function key(n, callback) {
function keys (line 7190) | function keys(callback) {
function executeCallback (line 7223) | function executeCallback(promise, callback) {
function _initStorage (line 7286) | function _initStorage(options) {
function clear (line 7313) | function clear(callback) {
function getItem (line 7334) | function getItem(key, callback) {
function iterate (line 7363) | function iterate(iterator, callback) {
function key (line 7408) | function key(n, callback) {
function keys (line 7431) | function keys(callback) {
function length (line 7452) | function length(callback) {
function removeItem (line 7463) | function removeItem(key, callback) {
function setItem (line 7485) | function setItem(key, value, callback) {
function executeCallback (line 7530) | function executeCallback(promise, callback) {
function _createBlob (line 7599) | function _createBlob(parts, properties) {
function serialize (line 7624) | function serialize(value, callback) {
function deserialize (line 7701) | function deserialize(value) {
function stringToBuffer (line 7755) | function stringToBuffer(serializedString) {
function bufferToString (line 7789) | function bufferToString(buffer) {
function _initStorage (line 7852) | function _initStorage(options) {
function getItem (line 7894) | function getItem(key, callback) {
function iterate (line 7929) | function iterate(iterator, callback) {
function setItem (line 7973) | function setItem(key, value, callback) {
function removeItem (line 8028) | function removeItem(key, callback) {
function clear (line 8057) | function clear(callback) {
function length (line 8079) | function length(callback) {
function key (line 8110) | function key(n, callback) {
function keys (line 8131) | function keys(callback) {
function executeCallback (line 8158) | function executeCallback(promise, callback) {
FILE: browser-version/test/async.js
function n (line 1) | function n(){}
function t (line 1) | function t(n){return n}
function e (line 1) | function e(n){return!!n}
function r (line 1) | function r(n){return!n}
function u (line 1) | function u(n){return function(){if(null===n)throw new Error("Callback wa...
function i (line 1) | function i(n){return function(){null!==n&&(n.apply(this,arguments),n=nul...
function o (line 1) | function o(n){return M(n)||"number"==typeof n.length&&n.length>=0&&n.len...
function c (line 1) | function c(n,t){for(var e=-1,r=n.length;++e<r;)t(n[e],e,n)}
function a (line 1) | function a(n,t){for(var e=-1,r=n.length,u=Array(r);++e<r;)u[e]=t(n[e],e,...
function f (line 1) | function f(n){return a(Array(n),function(n,t){return t})}
function l (line 1) | function l(n,t,e){return c(n,function(n,r,u){e=t(e,n,r,u)}),e}
function s (line 1) | function s(n,t){c(W(n),function(e){t(n[e],e)})}
function p (line 1) | function p(n,t){for(var e=0;e<n.length;e++)if(n[e]===t)return e;return-1}
function h (line 1) | function h(n){var t,e,r=-1;return o(n)?(t=n.length,function(){return r++...
function m (line 1) | function m(n,t){return t=null==t?n.length-1:+t,function(){for(var e=Math...
function y (line 1) | function y(n){return function(t,e,r){return n(t,r)}}
function v (line 1) | function v(t){return function(e,r,o){o=i(o||n),e=e||[];var c=h(e);if(0>=...
function d (line 1) | function d(n){return function(t,e,r){return n(C.eachOf,t,e,r)}}
function g (line 1) | function g(n){return function(t,e,r,u){return n(v(e),t,r,u)}}
function k (line 1) | function k(n){return function(t,e,r){return n(C.eachOfSeries,t,e,r)}}
function b (line 1) | function b(t,e,r,u){u=i(u||n),e=e||[];var c=o(e)?[]:{};t(e,function(n,t,...
function w (line 1) | function w(n,t,e,r){var u=[];n(t,function(n,t,r){e(n,function(e){e&&u.pu...
function O (line 1) | function O(n,t,e,r){w(n,t,function(n,t){e(n,function(n){t(!n)})},r)}
function S (line 1) | function S(n,t,e){return function(r,u,i,o){function c(){o&&o(e(!1,void 0...
function E (line 1) | function E(n,t){return t}
function L (line 1) | function L(t,e,r){r=r||n;var u=o(e)?[]:{};t(e,function(n,t,e){n(m(functi...
function I (line 1) | function I(n,t,e,r){var u=[];n(t,function(n,t,r){e(n,function(n,t){u=u.c...
function x (line 1) | function x(t,e,r){function i(t,e,r,u){if(null!=u&&"function"!=typeof u)t...
function j (line 1) | function j(n){return m(function(t,e){t.apply(null,e.concat([m(function(t...
function A (line 1) | function A(n){return function(t,e,r){n(f(t),e,r)}}
function T (line 1) | function T(n){return m(function(t,e){var r=m(function(e){var r=this,u=e....
function z (line 1) | function z(n){return m(function(t){var e=t.pop();t.push(function(){var n...
function o (line 1) | function o(n){f--,n?r(n):null===c&&0>=f&&r(null)}
function o (line 1) | function o(){var n=!0;return null===a?r(null):(e(t[a],a,u(function(t){if...
function r (line 1) | function r(n,t){var e=n.criteria,r=t.criteria;return r>e?-1:e>r?1:0}
function u (line 1) | function u(n){d.unshift(n)}
function o (line 1) | function o(n){var t=p(d,n);t>=0&&d.splice(t,1)}
function a (line 1) | function a(){h--,c(d.slice(0),function(n){n()})}
function i (line 1) | function i(){return e>v&&l(g,function(n,t){return n&&y.hasOwnProperty(t)...
function c (line 1) | function c(){i()&&(v++,o(c),h[h.length-1](d,y))}
function r (line 1) | function r(n,t){if("number"==typeof t)n.times=parseInt(t,10)||i;else{if(...
function u (line 1) | function u(n,t){function e(n,e){return function(r){n(function(n,t){r(!n|...
function r (line 1) | function r(n){return m(function(t,u){if(t)e.apply(null,[t].concat(u));el...
function t (line 1) | function t(e){function r(){return n.length&&n[e].apply(null,arguments),r...
function r (line 1) | function r(n,t){return n.priority-t.priority}
function u (line 1) | function u(n,t,e){for(var r=-1,u=n.length-1;u>r;){var i=r+(u-r+1>>>1);e(...
function i (line 1) | function i(t,e,i,o){if(null!=o&&"function"!=typeof o)throw new Error("ta...
function r (line 1) | function r(n){return n?i(n):void o(r)}
FILE: browser-version/test/chai.js
function require (line 12) | function require(name) {
function showError (line 67) | function showError(name) {
function exclude (line 157) | function exclude () {
function AssertionError (line 196) | function AssertionError (message, _props, ssf) {
function getType (line 292) | function getType (obj) {
function Library (line 314) | function Library () {
function deepEqual (line 441) | function deepEqual(a, b, m) {
function sameValue (line 471) | function sameValue(a, b) {
function typeEqual (line 487) | function typeEqual(a, b) {
function dateEqual (line 500) | function dateEqual(a, b) {
function regexpEqual (line 514) | function regexpEqual(a, b) {
function argumentsEqual (line 530) | function argumentsEqual(a, b, m) {
function enumerable (line 544) | function enumerable(a) {
function iterableEqual (line 559) | function iterableEqual(a, b) {
function bufferEqual (line 584) | function bufferEqual(a, b) {
function isValue (line 597) | function isValue(a) {
function objectEqual (line 612) | function objectEqual(a, b, m) {
function Assertion (line 796) | function Assertion (obj, msg, stack) {
function an (line 1096) | function an (type, msg) {
function includeChainingBehavior (line 1133) | function includeChainingBehavior () {
function include (line 1137) | function include (val, msg) {
function checkArguments (line 1346) | function checkArguments () {
function assertEqual (line 1381) | function assertEqual (val, msg) {
function assertEql (line 1417) | function assertEql(obj, msg) {
function assertAbove (line 1455) | function assertAbove (n, msg) {
function assertLeast (line 1503) | function assertLeast (n, msg) {
function assertBelow (line 1551) | function assertBelow (n, msg) {
function assertMost (line 1599) | function assertMost (n, msg) {
function assertInstanceOf (line 1685) | function assertInstanceOf (constructor, msg) {
function assertOwnProperty (line 1812) | function assertOwnProperty (name, msg) {
function assertLengthChain (line 1851) | function assertLengthChain () {
function assertLength (line 1855) | function assertLength (n, msg) {
function assertKeys (line 1960) | function assertKeys (keys) {
function assertThrows (line 2080) | function assertThrows (constructor, errMsg, msg) {
function isSubsetOf (line 2313) | function isSubsetOf(subset, superset, cmp) {
function assertChanges (line 2393) | function assertChanges (object, prop, msg) {
function assertIncreases (line 2430) | function assertIncreases (object, prop, msg) {
function assertDecreases (line 2467) | function assertDecreases (object, prop, msg) {
function loadShould (line 3778) | function loadShould () {
function parsePath (line 4352) | function parsePath (path) {
function _getPathValue (line 4379) | function _getPathValue (parsed, obj, index) {
function addProperty (line 4491) | function addProperty(property) {
function inspect (line 4659) | function inspect(obj, showHidden, depth, colors) {
function formatValue (line 4680) | function formatValue(ctx, value, recurseTimes) {
function formatPrimitive (line 4820) | function formatPrimitive(ctx, value) {
function formatError (line 4847) | function formatError(value) {
function formatArray (line 4852) | function formatArray(ctx, value, recurseTimes, visibleKeys, keys) {
function formatProperty (line 4872) | function formatProperty(ctx, value, recurseTimes, visibleKeys, key, arra...
function reduceToSingleString (line 4932) | function reduceToSingleString(output, base, braces) {
function isArray (line 4952) | function isArray(ar) {
function isRegExp (line 4957) | function isRegExp(re) {
function isDate (line 4961) | function isDate(d) {
function isError (line 4965) | function isError(e) {
function objectToString (line 4969) | function objectToString(o) {
FILE: browser-version/test/localforage.js
function resolve (line 44) | function resolve(child) {
function all (line 108) | function all(promises) {
function useNextTick (line 160) | function useNextTick() {
function useMutationObserver (line 166) | function useMutationObserver() {
function useSetTimeout (line 177) | function useSetTimeout() {
function flush (line 184) | function flush() {
function asap (line 204) | function asap(callback, arg) {
function configure (line 224) | function configure(name, value) {
function polyfill (line 243) | function polyfill() {
function Promise (line 296) | function Promise(resolver) {
function invokeResolver (line 310) | function invokeResolver(resolver, promise) {
function invokeCallback (line 326) | function invokeCallback(settled, promise, callback, detail) {
function subscribe (line 361) | function subscribe(parent, child, onFulfillment, onRejection) {
function publish (line 370) | function publish(promise, settled) {
function handleThenable (line 417) | function handleThenable(promise, value) {
function resolve (line 458) | function resolve(promise, value) {
function fulfill (line 466) | function fulfill(promise, value) {
function reject (line 474) | function reject(promise, reason) {
function publishFulfillment (line 482) | function publishFulfillment(promise) {
function publishRejection (line 486) | function publishRejection(promise) {
function race (line 552) | function race(promises) {
function reject (line 610) | function reject(reason) {
function resolve (line 625) | function resolve(value) {
function objectOrFunction (line 644) | function objectOrFunction(x) {
function isFunction (line 648) | function isFunction(x) {
function isArray (line 652) | function isArray(x) {
function __webpack_require__ (line 682) | function __webpack_require__(moduleId) {
function _classCallCheck (line 727) | function _classCallCheck(instance, Constructor) { if (!(instance instanc...
function callWhenReady (line 810) | function callWhenReady(localForageInstance, libraryMethod) {
function extend (line 819) | function extend() {
function isLibraryDriver (line 839) | function isLibraryDriver(driverName) {
function LocalForage (line 850) | function LocalForage(options) {
function setDriverToConfig (line 1031) | function setDriverToConfig() {
function initDriver (line 1035) | function initDriver(supportedDrivers) {
function _createBlob (line 1165) | function _createBlob(parts, properties) {
function _binStringToArrayBuffer (line 1188) | function _binStringToArrayBuffer(bin) {
function _blobAjax (line 1201) | function _blobAjax(url) {
function _checkBlobSupportWithoutCaching (line 1239) | function _checkBlobSupportWithoutCaching(idb) {
function _checkBlobSupport (line 1269) | function _checkBlobSupport(idb) {
function _encodeBlob (line 1280) | function _encodeBlob(blob) {
function _decodeBlob (line 1297) | function _decodeBlob(encodedBlob) {
function _isEncodedBlob (line 1303) | function _isEncodedBlob(value) {
function _initStorage (line 1309) | function _initStorage(options) {
function _getOriginalConnection (line 1392) | function _getOriginalConnection(dbInfo) {
function _getUpgradedConnection (line 1396) | function _getUpgradedConnection(dbInfo) {
function _getConnection (line 1400) | function _getConnection(dbInfo, upgradeNeeded) {
function _isUpgradeNeeded (line 1447) | function _isUpgradeNeeded(dbInfo, defaultVersion) {
function getItem (line 1483) | function getItem(key, callback) {
function iterate (line 1520) | function iterate(iterator, callback) {
function setItem (line 1562) | function setItem(key, value, callback) {
function removeItem (line 1618) | function removeItem(key, callback) {
function clear (line 1660) | function clear(callback) {
function length (line 1685) | function length(callback) {
function key (line 1708) | function key(n, callback) {
function keys (line 1760) | function keys(callback) {
function executeCallback (line 1793) | function executeCallback(promise, callback) {
function _initStorage (line 1856) | function _initStorage(options) {
function clear (line 1883) | function clear(callback) {
function getItem (line 1904) | function getItem(key, callback) {
function iterate (line 1933) | function iterate(iterator, callback) {
function key (line 1978) | function key(n, callback) {
function keys (line 2001) | function keys(callback) {
function length (line 2022) | function length(callback) {
function removeItem (line 2033) | function removeItem(key, callback) {
function setItem (line 2055) | function setItem(key, value, callback) {
function executeCallback (line 2100) | function executeCallback(promise, callback) {
function _createBlob (line 2169) | function _createBlob(parts, properties) {
function serialize (line 2194) | function serialize(value, callback) {
function deserialize (line 2271) | function deserialize(value) {
function stringToBuffer (line 2325) | function stringToBuffer(serializedString) {
function bufferToString (line 2359) | function bufferToString(buffer) {
function _initStorage (line 2422) | function _initStorage(options) {
function getItem (line 2464) | function getItem(key, callback) {
function iterate (line 2499) | function iterate(iterator, callback) {
function setItem (line 2543) | function setItem(key, value, callback) {
function removeItem (line 2598) | function removeItem(key, callback) {
function clear (line 2627) | function clear(callback) {
function length (line 2649) | function length(callback) {
function key (line 2680) | function key(n, callback) {
function keys (line 2701) | function keys(callback) {
function executeCallback (line 2728) | function executeCallback(promise, callback) {
FILE: browser-version/test/mocha.js
function require (line 6) | function require(p){
function isArray (line 76) | function isArray(obj) {
function EventEmitter (line 86) | function EventEmitter(){}
function on (line 121) | function on () {
function Progress (line 265) | function Progress() {
function Context (line 407) | function Context(){}
function Hook (line 488) | function Hook(title, fn) {
function visit (line 690) | function visit(obj) {
function image (line 987) | function image(name) {
function Mocha (line 1008) | function Mocha(options) {
function parse (line 1269) | function parse(str) {
function format (line 1308) | function format(ms) {
function Base (line 1519) | function Base(runner) {
function pluralize (line 1585) | function pluralize(n) {
function pad (line 1635) | function pad(str, len) {
function errorDiff (line 1648) | function errorDiff(err, type) {
function colorLines (line 1669) | function colorLines(name, str) {
function Doc (line 1699) | function Doc(runner) {
function Dot (line 1759) | function Dot(runner) {
function HTMLCov (line 1827) | function HTMLCov(runner) {
function coverageClass (line 1851) | function coverageClass(n) {
function HTML (line 1904) | function HTML(runner, root) {
function error (line 2040) | function error(msg) {
function fragment (line 2048) | function fragment(html) {
function text (line 2067) | function text(el, str) {
function on (line 2079) | function on(el, event, fn) {
function JSONCov (line 2134) | function JSONCov(runner, output) {
function map (line 2177) | function map(cov) {
function coverage (line 2212) | function coverage(filename, data) {
function clean (line 2255) | function clean(test) {
function List (line 2287) | function List(runner) {
function clean (line 2320) | function clean(test) {
function JSONReporter (line 2352) | function JSONReporter(runner) {
function clean (line 2393) | function clean(test) {
function Landing (line 2443) | function Landing(runner) {
function List (line 2527) | function List(runner) {
function Markdown (line 2595) | function Markdown(runner) {
function Min (line 2690) | function Min(runner) {
function NyanCat (line 2734) | function NyanCat(runner) {
function draw (line 2800) | function draw(color, n) {
function write (line 2963) | function write(string) {
function Progress (line 3007) | function Progress(runner, options) {
function Spec (line 3092) | function Spec(runner) {
function TAP (line 3185) | function TAP(runner) {
function title (line 3223) | function title(test) {
function Teamcity (line 3250) | function Teamcity(runner) {
function escape (line 3283) | function escape(str) {
function XUnit (line 3331) | function XUnit(runner) {
function test (line 3373) | function test(test) {
function tag (line 3395) | function tag(name, attrs, close, content) {
function cdata (line 3413) | function cdata(str) {
function Runnable (line 3452) | function Runnable(title, fn) {
function multiple (line 3586) | function multiple(err) {
function done (line 3593) | function done(err) {
function Runner (line 3670) | function Runner(suite) {
function next (line 3815) | function next(i) {
function next (line 3855) | function next(suite) {
function next (line 3952) | function next(err) {
function next (line 4019) | function next() {
function done (line 4025) | function done() {
function uncaught (line 4080) | function uncaught(err) {
function filterLeaks (line 4112) | function filterLeaks(ok) {
function Suite (line 4173) | function Suite(title, ctx) {
function Test (line 4448) | function Test(title, fn) {
function ignored (line 4610) | function ignored(path){
function highlight (line 4722) | function highlight(js) {
FILE: browser-version/test/nedb-browser.js
function findById (line 13) | function findById (docs, id) {
FILE: browser-version/test/testLoad.js
function someInserts (line 10) | function someInserts (sn, N, callback) {
function someLS (line 23) | function someLS (sn, N, callback) {
function someLSDiff (line 33) | function someLSDiff (sn, N, callback) {
function someLF (line 43) | function someLF (sn, N, callback) {
function someLFDiff (line 59) | function someLFDiff (sn, N, callback) {
FILE: browser-version/test/testPersistence.js
function testsFailed (line 4) | function testsFailed () {
FILE: browser-version/test/testPersistence2.js
function testsFailed (line 13) | function testsFailed () {
FILE: lib/cursor.js
function Cursor (line 16) | function Cursor (db, query, execFn) {
function callback (line 124) | function callback (error, res) {
FILE: lib/customUtils.js
function uid (line 12) | function uid (len) {
FILE: lib/datastore.js
function Datastore (line 29) | function Datastore (options) {
FILE: lib/executor.js
function Executor (line 8) | function Executor () {
FILE: lib/indexes.js
function checkValueEquality (line 10) | function checkValueEquality (a, b) {
function projectForUnique (line 17) | function projectForUnique (elt) {
function Index (line 36) | function Index (options) {
FILE: lib/model.js
function checkKey (line 26) | function checkKey (k, v) {
function checkObject (line 45) | function checkObject (obj) {
function serialize (line 69) | function serialize (obj) {
function deserialize (line 93) | function deserialize (rawData) {
function deepCopy (line 109) | function deepCopy (obj, strictKeys) {
function isPrimitiveType (line 144) | function isPrimitiveType (obj) {
function compareNSB (line 159) | function compareNSB (a, b) {
function compareArrays (line 165) | function compareArrays (a, b) {
function compareThings (line 189) | function compareThings (a, b, _compareStrings) {
function createModifierFunction (line 412) | function createModifierFunction (modifier) {
function modify (line 437) | function modify (obj, updateQuery) {
function getDotValue (line 493) | function getDotValue (obj, field) {
function areThingsEqual (line 528) | function areThingsEqual (a, b) {
function areComparable (line 563) | function areComparable (a, b) {
function match (line 734) | function match (obj, query) {
function matchQueryPart (line 766) | function matchQueryPart (obj, queryKey, queryValue, treatObjAsValue) {
FILE: lib/persistence.js
function Persistence (line 23) | function Persistence (options) {
FILE: test/db.test.js
function onload (line 84) | function onload (err) {
function testPostUpdateState (line 1101) | function testPostUpdateState (cb) {
function testPostUpdateState (line 1159) | function testPostUpdateState (cb) {
function testPostUpdateState (line 1739) | function testPostUpdateState (cb) {
FILE: test/executor.test.js
function testThrowInCallback (line 16) | function testThrowInCallback (d, done) {
function testFalsyCallback (line 42) | function testFalsyCallback (d, done) {
function testRightOrder (line 63) | function testRightOrder (d, done) {
function testExecutorWorksWithoutCallback (line 118) | function testExecutorWorksWithoutCallback (d, done) {
FILE: test_lac/loadAndCrash.test.js
function rethrow (line 6) | function rethrow() {
function maybeCallback (line 27) | function maybeCallback(cb) {
function isFd (line 31) | function isFd(path) {
function assertEncoding (line 35) | function assertEncoding(encoding) {
function writeAll (line 42) | function writeAll(fd, isUserFd, buffer, offset, length, position, callba...
function writeFd (line 106) | function writeFd(fd, isUserFd) {
FILE: test_lac/openFds.test.js
function multipleOpen (line 10) | function multipleOpen (filename, N, callback) {
Condensed preview — 55 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (1,141K chars).
[
{
"path": ".gitignore",
"chars": 177,
"preview": "lib-cov\n*.seed\n*.log\n*.csv\n*.dat\n*.out\n*.pid\n*.gz\n.idea\npids\nlogs\nresults\n\nnpm-debug.log\nworkspace\nnode_modules\n\nbrowser"
},
{
"path": "LICENSE",
"chars": 1109,
"preview": "(The MIT License)\n\nCopyright (c) 2013 Louis Chatriot <louis.chatriot@gmail.com>\n\nPermission is hereby granted, fre"
},
{
"path": "README.md",
"chars": 41324,
"preview": "<img src=\"http://i.imgur.com/9O1xHFb.png\" style=\"width: 25%; height: 25%; float: left;\">\n\n## The JavaScript Database\n\n> "
},
{
"path": "benchmarks/commonUtilities.js",
"chars": 8262,
"preview": "/**\n * Functions that are used in several benchmark tests\n */\n\nvar customUtils = require('../lib/customUtils')\n , fs = "
},
{
"path": "benchmarks/ensureIndex.js",
"chars": 1480,
"preview": "var Datastore = require('../lib/datastore')\n , benchDb = 'workspace/insert.bench.db'\n , async = require('async')\n , c"
},
{
"path": "benchmarks/find.js",
"chars": 958,
"preview": "var Datastore = require('../lib/datastore')\n , benchDb = 'workspace/find.bench.db'\n , fs = require('fs')\n , path = re"
},
{
"path": "benchmarks/findOne.js",
"chars": 1025,
"preview": "var Datastore = require('../lib/datastore')\n , benchDb = 'workspace/findOne.bench.db'\n , fs = require('fs')\n , path ="
},
{
"path": "benchmarks/findWithIn.js",
"chars": 964,
"preview": "var Datastore = require('../lib/datastore')\n , benchDb = 'workspace/find.bench.db'\n , fs = require('fs')\n , path = re"
},
{
"path": "benchmarks/insert.js",
"chars": 1159,
"preview": "var Datastore = require('../lib/datastore')\n , benchDb = 'workspace/insert.bench.db'\n , async = require('async')\n , e"
},
{
"path": "benchmarks/loadDatabase.js",
"chars": 1203,
"preview": "var Datastore = require('../lib/datastore')\n , benchDb = 'workspace/loaddb.bench.db'\n , fs = require('fs')\n , path = "
},
{
"path": "benchmarks/remove.js",
"chars": 1394,
"preview": "var Datastore = require('../lib/datastore')\n , benchDb = 'workspace/remove.bench.db'\n , fs = require('fs')\n , path = "
},
{
"path": "benchmarks/update.js",
"chars": 1398,
"preview": "var Datastore = require('../lib/datastore')\n , benchDb = 'workspace/update.bench.db'\n , fs = require('fs')\n , path = "
},
{
"path": "bower.json",
"chars": 223,
"preview": "{\n\"name\": \"nedb\",\n\"description\": \"The Javascript Database for Node, nwjs, Electron and the browser\",\n\"ignore\": [\"benchma"
},
{
"path": "browser-version/browser-specific/lib/customUtils.js",
"chars": 2415,
"preview": "/**\n * Specific customUtils for the browser, where we don't have access to the Crypto and Buffer modules\n */\n\n/**\n * Tak"
},
{
"path": "browser-version/browser-specific/lib/storage.js",
"chars": 2841,
"preview": "/**\n * Way data is stored for this database\n * For a Node.js/Node Webkit database it's the file system\n * For a browser-"
},
{
"path": "browser-version/build.js",
"chars": 2973,
"preview": "/**\n * Build the browser version of nedb\n */\n\nvar fs = require('fs')\n , path = require('path')\n , child_process = requ"
},
{
"path": "browser-version/out/nedb.js",
"chars": 299286,
"preview": "(function(e){if(\"function\"==typeof bootstrap)bootstrap(\"nedb\",e);else if(\"object\"==typeof exports)module.exports=e();els"
},
{
"path": "browser-version/package.json",
"chars": 132,
"preview": "{\n \"dependencies\": {\n \"async\": \"~0.2.9\",\n \"fs-extra\": \"~0.6.3\",\n \"uglify-js\": \"~2.3.6\",\n \"browserify\": \"~2."
},
{
"path": "browser-version/test/async.js",
"chars": 12340,
"preview": "!function(){function n(){}function t(n){return n}function e(n){return!!n}function r(n){return!n}function u(n){return fun"
},
{
"path": "browser-version/test/chai.js",
"chars": 135717,
"preview": "\n;(function(){\n\n/**\n * Require the module at `name`.\n *\n * @param {String} name\n * @return {Object} exports\n * @api publ"
},
{
"path": "browser-version/test/index.html",
"chars": 606,
"preview": "<!DOCTYPE html>\n<html>\n<head>\n <meta charset=\"utf-8\">\n <title>Mocha tests for NeDB</title>\n <link rel=\"stylesheet\" hr"
},
{
"path": "browser-version/test/localforage.js",
"chars": 96550,
"preview": "/*!\n localForage -- Offline Storage, Improved\n Version 1.3.0\n https://mozilla.github.io/localForage\n (c) 201"
},
{
"path": "browser-version/test/mocha.css",
"chars": 2958,
"preview": "@charset \"UTF-8\";\nbody {\n font: 20px/1.5 \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n padding: 60px 50px;\n}\n\n#moch"
},
{
"path": "browser-version/test/mocha.js",
"chars": 96172,
"preview": ";(function(){\n\n\n// CommonJS require()\n\nfunction require(p){\n var path = require.resolve(p)\n , mod = require.modu"
},
{
"path": "browser-version/test/nedb-browser.js",
"chars": 10112,
"preview": "/**\n * Testing the browser version of NeDB\n * The goal of these tests is not to be exhaustive, we have the server-side N"
},
{
"path": "browser-version/test/playground.html",
"chars": 169,
"preview": "<!DOCTYPE html>\n<html>\n<head>\n <meta charset=\"utf-8\">\n <title>Playground for NeDB</title>\n</head>\n<body>\n <script src"
},
{
"path": "browser-version/test/testLoad.html",
"chars": 377,
"preview": "<!DOCTYPE html>\n<html>\n<head>\n <meta charset=\"utf-8\">\n <title>Test NeDB persistence load in the browser</title>\n <lin"
},
{
"path": "browser-version/test/testLoad.js",
"chars": 4352,
"preview": "console.log('BEGINNING');\n\nvar N = 50000\n , db = new Nedb({ filename: 'loadTest', autoload: true })\n , t, i\n , sample"
},
{
"path": "browser-version/test/testPersistence.html",
"chars": 298,
"preview": "<!DOCTYPE html>\n<html>\n<head>\n <meta charset=\"utf-8\">\n <title>Test NeDB persistence in the browser</title>\n <link rel"
},
{
"path": "browser-version/test/testPersistence.js",
"chars": 589,
"preview": "console.log(\"Beginning tests\");\nconsole.log(\"Please note these tests work on Chrome latest, might not work on other brow"
},
{
"path": "browser-version/test/testPersistence2.html",
"chars": 349,
"preview": "<!DOCTYPE html>\n<html>\n<head>\n <meta charset=\"utf-8\">\n <title>Test NeDB persistence in the browser - Results</title>\n "
},
{
"path": "browser-version/test/testPersistence2.js",
"chars": 1116,
"preview": "// Capture F5 to reload the base page testPersistence.html not this one\n$(document).on('keydown', function (e) {\n if (e"
},
{
"path": "index.js",
"chars": 73,
"preview": "var Datastore = require('./lib/datastore');\n\nmodule.exports = Datastore;\n"
},
{
"path": "lib/cursor.js",
"chars": 5390,
"preview": "/**\n * Manage access to data, be it to find, update or remove it\n */\nvar model = require('./model')\n , _ = require('und"
},
{
"path": "lib/customUtils.js",
"chars": 667,
"preview": "var crypto = require('crypto')\n ;\n\n/**\n * Return a random alphanumerical string of length len\n * There is a very small "
},
{
"path": "lib/datastore.js",
"chars": 23806,
"preview": "var customUtils = require('./customUtils')\n , model = require('./model')\n , async = require('async')\n , Executor = re"
},
{
"path": "lib/executor.js",
"chars": 2472,
"preview": "/**\n * Responsible for sequentially executing actions on the database\n */\n\nvar async = require('async')\n ;\n\nfunction Ex"
},
{
"path": "lib/indexes.js",
"chars": 6986,
"preview": "var BinarySearchTree = require('binary-search-tree').AVLTree\n , model = require('./model')\n , _ = require('underscore'"
},
{
"path": "lib/model.js",
"chars": 25736,
"preview": "/**\n * Handle models (i.e. docs)\n * Serialization/deserialization\n * Copying\n * Querying, update\n */\n\nvar util = require"
},
{
"path": "lib/persistence.js",
"chars": 10847,
"preview": "/**\n * Handle every persistence-related task\n * The interface Datastore expects to be implemented is\n * * Persistence.lo"
},
{
"path": "lib/storage.js",
"chars": 4209,
"preview": "/**\n * Way data is stored for this database\n * For a Node.js/Node Webkit database it's the file system\n * For a browser-"
},
{
"path": "package.json",
"chars": 1121,
"preview": "{\n \"name\": \"nedb\",\n \"version\": \"1.8.0\",\n \"author\": {\n \"name\": \"Louis Chatriot\",\n \"email\": \"louis.chatriot@gmail"
},
{
"path": "test/cursor.test.js",
"chars": 29071,
"preview": "var should = require('chai').should()\n , assert = require('chai').assert\n , testDb = 'workspace/test.db'\n , fs = requ"
},
{
"path": "test/customUtil.test.js",
"chars": 693,
"preview": "var should = require('chai').should()\n , assert = require('chai').assert\n , customUtils = require('../lib/customUtils'"
},
{
"path": "test/db.test.js",
"chars": 112880,
"preview": "var should = require('chai').should()\n , assert = require('chai').assert\n , testDb = 'workspace/test.db'\n , fs = requ"
},
{
"path": "test/executor.test.js",
"chars": 6501,
"preview": "var should = require('chai').should()\n , assert = require('chai').assert\n , testDb = 'workspace/test.db'\n , fs = requ"
},
{
"path": "test/indexes.test.js",
"chars": 27974,
"preview": "var Index = require('../lib/indexes')\n , customUtils = require('../lib/customUtils')\n , should = require('chai').shoul"
},
{
"path": "test/mocha.opts",
"chars": 31,
"preview": "--reporter spec\n--timeout 30000"
},
{
"path": "test/model.test.js",
"chars": 66927,
"preview": "var model = require('../lib/model')\n , should = require('chai').should()\n , assert = require('chai').assert\n , expect"
},
{
"path": "test/persistence.test.js",
"chars": 35557,
"preview": "var should = require('chai').should()\n , assert = require('chai').assert\n , testDb = 'workspace/test.db'\n , fs = requ"
},
{
"path": "test_lac/loadAndCrash.test.js",
"chars": 3346,
"preview": "/**\n * Load and modify part of fs to ensure writeFile will crash after writing 5000 bytes\n */\nvar fs = require('fs');\n\nf"
},
{
"path": "test_lac/openFds.test.js",
"chars": 2144,
"preview": "var fs = require('fs')\n , child_process = require('child_process')\n , async = require('async')\n , Nedb = require('../"
},
{
"path": "test_lac/openFdsLaunch.sh",
"chars": 46,
"preview": "ulimit -n 128\nnode ./test_lac/openFds.test.js\n"
},
{
"path": "test_lac/openFdsTestFile",
"chars": 13,
"preview": "Random stuff\n"
},
{
"path": "test_lac/openFdsTestFile2",
"chars": 24,
"preview": "Some other random stuff\n"
}
]
About this extraction
This page contains the full source code of the louischatriot/nedb GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 55 files (1.0 MB), approximately 274.8k tokens, and a symbol index with 454 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.