Repository: FranciscoP/drive-db
Branch: master
Commit: 6ea4268aa081
Files: 9
Total size: 14.8 KB

Directory structure:
gitextract_4tqei2hx/

├── .github/
│   ├── FUNDING.yml
│   └── workflows/
│       └── tests.yml
├── .gitignore
├── LICENSE
├── README.md
├── demo.js
├── index.js
├── index.test.js
└── package.json

================================================
FILE CONTENTS
================================================

================================================
FILE: .github/FUNDING.yml
================================================
custom: https://www.paypal.me/franciscopresencia/19


================================================
FILE: .github/workflows/tests.yml
================================================
name: tests

on: [push]

jobs:
  build:

    runs-on: ubuntu-latest

    strategy:
      matrix:
        node-version: [8.x, 10.x, 12.x]

    steps:
    - uses: actions/checkout@v1
    - name: Use Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v1
      with:
        node-version: ${{ matrix.node-version }}
    - name: install dependencies
      run: npm install
    - name: npm test
      run: npm test
      env:
        CI: true


================================================
FILE: .gitignore
================================================
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*

# Temporary folder
/temp

# Mac temporal file
.DS_Store

# SASS Cache
.sass-cache

# Runtime data
pids
*.pid
*.seed
*.pid.lock

# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov

# Coverage directory used by tools like istanbul
coverage

# nyc test coverage
.nyc_output

# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
.grunt

# Bower dependency directory (https://bower.io/)
bower_components

# node-waf configuration
.lock-wscript

# Compiled binary addons (https://nodejs.org/api/addons.html)
build/Release

# Dependency directories
node_modules/
jspm_packages/

# TypeScript v1 declaration files
typings/

# Optional npm cache directory
.npm

# Optional eslint cache
.eslintcache

# Optional REPL history
.node_repl_history

# Output of 'npm pack'
*.tgz

# Yarn Integrity file
.yarn-integrity

# dotenv environment variables file
.env

# next.js build output
.next

# This is a library so don't include it
package-lock.json


================================================
FILE: LICENSE
================================================
The MIT License (MIT)

Copyright (c) 2015 Francisco Presencia

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.



================================================
FILE: README.md
================================================
# drive-db [![npm install drive-db](https://img.shields.io/badge/npm%20install-drive--db-blue.svg)](https://www.npmjs.com/package/drive-db) [![test badge](https://github.com/franciscop/drive-db/workflows/tests/badge.svg)](https://github.com/franciscop/drive-db/blob/master/index.test.js) [![demo](https://img.shields.io/badge/demo-blue.svg)](https://jsfiddle.net/franciscop/1w4t7mc5/)

> NOTICE: **Google has killed Google Sheets API v3 that allowed this library to exist**. So unfortunately this library no longer works. Their new API **does not** provide the same functionality and it doesn't allow you to publish a spreadshit, so we cannot do anything on our end.
>
> If you depended on this library for anything important, we are very sorry. The new Sheets v4 does work with you creating a whole backend-frontend OAuth workflow (which is what this library was designed to avoid), so please use other libraries out there that allow for this.
>
> Thank you so much everyone for using this library for as long as you have and the feedback and love sent. It is one of the first and favorite libraries I've ever made, and it has powered all sort of things AFAIK, from blog posts, public charts, Covid19 panels, etc. So it is with a heavy heart that I have to write this notice, deprecating `drive-db`.

Use Google Drive spreadsheets as a simple database for Node.js and the browser. Perfect for collaboration with multiple people editing the same spreadsheet:

```js
import drive from 'drive-db';

// Load the data from the Drive Spreadsheet
(async () => {
  const db = await drive("1fvz34wY6phWDJsuIneqvOoZRPfo6CfJyPg1BYgHt59k");
  console.log(db);
})();
```

| id | firstname | lastname | age | city          |
|----|-----------|----------|-----|---------------|
| 1  | John      | Smith    | 34  | San Francisco |
| 2  | Merry     | Johnson  | 19  | Tokyo         |
| 3  | Peter     | Williams | 45  | London        |

Becomes an array of objects with the corresponding keys:

```json
[
  {
    "id": "1",
    "firstname": "John",
    "lastname": "Smith",
    "age": "34",
    "city": "San Francisco"
  },
  {
    "id": "2",
    "firstname": "Merry",
    "lastname": "Johnson",
    "age": "19",
    "city": "Tokyo"
  },
  ...
]
```



## Getting Started

Create the Google Drive spreadsheet and **publish it**:

- Create [a Google Spreadsheet](https://www.google.com/sheets/about/)
- File > Publish to the Web > Publish
- Copy the id between `/spreadsheets/` and `/edit` in the url:

    > [https://docs.google.com/spreadsheets/d/<strong>1fvz34wY6phWDJsuIneqvOoZRPfo6CfJyPg1BYgHt59k</strong>/edit](https://docs.google.com/spreadsheets/d/1fvz34wY6phWDJsuIneqvOoZRPfo6CfJyPg1BYgHt59k/edit)

Now you can either add the CDN `<script>` or use a bundler. For the CDN:

```html
<script src="https://cdn.jsdelivr.net/npm/drive-db"></script>
```

Otherwise install `drive-db` in your project:

```bash
npm install drive-db
```

And then load the spreadsheet into your project:

```js
// Include the module and tell it which spreadsheet to use
const drive = require("drive-db");

// Create an async context to be able to call `await`
(async () => {
  // Load the data from the Drive Spreadsheet
  const db = await drive("1fvz34wY6phWDJsuIneqvOoZRPfo6CfJyPg1BYgHt59k");

  console.log(db);
})();
```

The table has to have a structure similar to this, where the first row are the alphanumeric field names:

| id | firstname | lastname | age | city          |
|----|-----------|----------|-----|---------------|
| 1  | John      | Smith    | 34  | San Francisco |
| 2  | Merry     | Johnson  | 19  | Tokyo         |
| 3  | Peter     | Williams | 45  | London        |

See [this document](https://docs.google.com/spreadsheets/d/1fvz34wY6phWDJsuIneqvOoZRPfo6CfJyPg1BYgHt59k/edit#gid=0) as an example. **Please do not request access to edit it**.



## API

You import a single default export depending on your configuration:

```js
// For ES7 modules
import drive from "drive-db";

// For common.js imports
const drive = require("drive-db");
```

To retrieve the data call it and await for the promise it returns:

```js
// With async/await:
const db = await drive(SHEET_ID);
const db = await drive(options);
console.log(db);

// Use the callback syntax:
drive(SHEET_ID).then(db => console.log(db));
drive(options).then(db => console.log(db));
```

**SHEET_ID**: alias of `options = { sheet: SHEET_ID }`:

```js
const db = await drive("1fvz34wY6phWDJsuIneqvOoZRPfo6CfJyPg1BYgHt59k");
console.log(db);
```

**options**: a simple object containing some options:

```js
const db = await drive({
  sheet: "1fvz34wY6phWDJsuIneqvOoZRPfo6CfJyPg1BYgHt59k",
  tab: "1",
  cache: 3600
});
```

- `sheet` (required): when editing a google spreadsheet, it's the part between `/spreadsheets/` and `/edit` in the url. Please make sure to also publish the spreadsheet before copying it (File > Publish to the Web > Publish)
- `tab` (`"1"`): the tab to use in the spreadsheet, which defaults to the first tab. It's the number *as a string* of the tab. See [this demo](https://jsfiddle.net/franciscop/oj0fg9n6/) as an example of how to load the second tab.
- `cache` (`3600`): set the maximum time (in **seconds**) that the current cache is valid. After this, the data will be loaded again when the function is called. This is really useful when combined with development env constant. Set to 0 to refresh in each request.

It returns a plain Javascript array. With ES6+, operations on arrays are great, but feel free to use Lodash or similar if you want some more advanced queries.

If you are using [server for Node.js](https://serverjs.io/) with ES6+:

```js
const drive = require("drive-db");
const sheet = "1fvz34wY6phWDJsuIneqvOoZRPfo6CfJyPg1BYgHt59k"; // Or from .env

const server = require("server");
const { get } = server.router;
const { render } = server.reply;

const home = get("/", async ctx => {
  const records = await drive(sheet);
  return render("index", { records });
});

server(home);
```

## Advanced

There are some more advanced things that you might consider. While I recommend you to read the code for this, here are a couple of examples.


### Warm the cache

To warm the cache on project launch before the first request comes in, call the promise without awaiting or doing anything with it:

```js
const drive = require("drive-db");
const sheet = "1fvz34wY6phWDJsuIneqvOoZRPfo6CfJyPg1BYgHt59k";
// Warms the cache as soon as the Node.js project is launched
drive(sheet);

const server = require("server");
const { get } = server.router;
const { render } = server.reply;

const home = get("/", async ctx => {
  // Cache is already warm when this request happens
  const records = await drive(sheet);
  return render("index", { records });
});

server(home);
```


### Refresh the cache

To force-refresh the cache at any point you can call `drive()` with a cache time of 0:

```js
drive({ sheet, cache: 0 });
```

### Load locally

This is not available anymore. Since `drive-db` returns a plain array, you can just load the data locally:

```js
const data = require("./data.json");
```



## Thanks to

- [Creating and publishing a node.js module](https://quickleft.com/blog/creating-and-publishing-a-node-js-module/)


================================================
FILE: demo.js
================================================
const drive = require(".");

(async () => {
  const db = await drive("1fvz34wY6phWDJsuIneqvOoZRPfo6CfJyPg1BYgHt59k");
  console.log(db);
})();


================================================
FILE: index.js
================================================
// Find the Google Drive data
const getKeys = row => Object.keys(row).filter(key => /^gsx\$/.test(key));
const parseRow = row => {
  return getKeys(row).reduce((obj, key) => {
    obj[key.slice(4)] = row[key].$t;
    return obj;
  }, {});
};

// Make a GET HTTP response and parse the JSON response
const get = async url => {
  // Try first with fetch() - browser, worker, polyfilled, etc
  if (typeof fetch !== "undefined") {
    const res = await fetch(url);
    if (!res.ok) throw new Error(`Error ${res.status} retrieving ${url}`);
    return res.json();
  }

  // Now try with Node.js, which needs to be promisified
  if (typeof require !== "undefined") {
    return new Promise((resolve, reject) => {
      const handler = res => {
        res.setEncoding("utf8");
        if (res.statusCode < 200 || res.statusCode >= 300) {
          return reject(new Error(`Error ${res.statusCode} retrieving ${url}`));
        }
        let data = "";
        res.on("data", chunk => (data += chunk));
        res.on("end", () => resolve(JSON.parse(data)));
      };
      require("https")
        .get(url, handler)
        .on("error", reject);
    });
  }

  // No supported method was found, display the warning
  throw new Error("fetch() is not available, please polyfill it");
};

const retrieve = async ({ sheet, tab }) => {
  const raw = await get(
    `https://spreadsheets.google.com/feeds/list/${sheet}/${tab}/public/values?alt=json`
  );
  return raw.feed.entry.map(parseRow);
};

// Memoize a callback similar to React
const memo = (cb, map = {}) => async (options, timeout) => {
  const key = JSON.stringify(options);
  const time = new Date();
  if (map[key] && time - map[key].time < timeout) {
    return map[key].value;
  }
  map[key] = { value: await cb(options), time };
  return map[key].value;
};

// It should be memoized here, since we memoize the whole *request* and *parse*
const getSheet = memo(retrieve);

// The main drive() function
export default async options => {
  const optObject = typeof options === "object" ? options : { sheet: options };
  const { sheet = "", tab = "default", cache = 3600 } = optObject;

  // To update the data we need to make sure we're working with an id
  if (!sheet) throw new Error("Need a Google Drive sheet id to load");

  return getSheet({ sheet, tab }, cache);
};


================================================
FILE: index.test.js
================================================
/* jshint expr:true */
import drive from "./index.js";

const https = jest.genMockFromModule("https");

const delay = time => new Promise(done => setTimeout(done, time));

// Testing that we are able to load the library
describe("drive-db", function() {
  it("returns a table", async () => {
    const db = await drive("1fvz34wY6phWDJsuIneqvOoZRPfo6CfJyPg1BYgHt59k");
    expect(Array.isArray(db)).toBe(true);
    expect(db.length).toBe(6);
    expect(db[0]).toEqual({
      id: "1",
      firstname: "John",
      lastname: "Smith",
      age: "34",
      city: "San Francisco",
      country: "USA",
      timestamp: "12/10/2010"
    });
  });

  it("requires a sheet", async () => {
    await expect(drive()).rejects.toEqual(
      new Error("Need a Google Drive sheet id to load")
    );
  });

  it("throws with a wrong sheet id", async () => {
    await expect(drive("abc")).rejects.toEqual(
      new Error(
        "Error 400 retrieving https://spreadsheets.google.com/feeds/list/abc/default/public/values?alt=json"
      )
    );
  });

  it("can work with old cache", async () => {
    const db = await drive("1fvz34wY6phWDJsuIneqvOoZRPfo6CfJyPg1BYgHt59k");
    expect(Array.isArray(db)).toBe(true);
    expect(db.length).toBe(6);
    const db2 = await drive("1fvz34wY6phWDJsuIneqvOoZRPfo6CfJyPg1BYgHt59k");
  });

  it("can work with expired cache", async () => {
    const db = await drive("1fvz34wY6phWDJsuIneqvOoZRPfo6CfJyPg1BYgHt59k");
    expect(Array.isArray(db)).toBe(true);
    expect(db.length).toBe(6);
    await delay(1000);
    const db2 = await drive("1fvz34wY6phWDJsuIneqvOoZRPfo6CfJyPg1BYgHt59k");
  });
});


================================================
FILE: package.json
================================================
{
  "name": "drive-db",
  "version": "6.0.1",
  "description": "📊 Use Google Drive spreadsheets as a simple database",
  "homepage": "https://github.com/franciscop/drive-db#readme",
  "repository": "https://github.com/franciscop/drive-db.git",
  "bugs": "https://github.com/franciscop/drive-db/issues",
  "funding": "https://www.paypal.me/franciscopresencia/19",
  "author": "Francisco Presencia <public@francisco.io> (https://francisco.io/)",
  "license": "MIT",
  "scripts": {
    "build": "rollup index.js --name drive --output.format umd | uglifyjs -o index.min.js",
    "demo": "node ./demo.js",
    "start": "jest --watch",
    "test": "jest --coverage"
  },
  "keywords": [
    "google",
    "drive",
    "database",
    "db",
    "table",
    "spreadsheet",
    "mongodb",
    "async"
  ],
  "main": "index.min.js",
  "files": [],
  "dependencies": {},
  "devDependencies": {
    "@babel/core": "^7.4.4",
    "@babel/preset-env": "^7.4.4",
    "babel-jest": "^24.8.0",
    "jest": "^24.8.0",
    "rollup": "^1.11.3",
    "uglify-es": "^3.3.9"
  },
  "babel": {
    "presets": [
      [
        "@babel/preset-env",
        {
          "targets": {
            "node": "current"
          }
        }
      ]
    ]
  }
}