Full Code of kimmobrunfeldt/url-to-image for AI

Repository: kimmobrunfeldt/url-to-image
Branch: master
Commit: 0a75f3308540
Files: 11
Total size: 25.3 KB

Directory structure:
gitextract_4al94_04/

├── .editorconfig
├── .gitignore
├── .travis.yml
├── CHANGELOG.md
├── LICENSE
├── README.md
├── package.json
├── src/
│   ├── cli-parser.js
│   ├── index.js
│   └── url-to-image.js
└── test/
    └── test-functional.js

================================================
FILE CONTENTS
================================================

================================================
FILE: .editorconfig
================================================
# EditorConfig is awesome: http://EditorConfig.org

# top-most EditorConfig file
root = true

# Unix-style newlines with a newline ending every file
[*]
end_of_line = lf
insert_final_newline = true

# 4 space indentation
[*.py]
indent_style = space
indent_size = 4

# Tab indentation (no size specified)
[*.js]
indent_style = space
indent_size = 4

# Matches the exact files package.json and .travis.yml
[{package.json,.travis.yml,Gruntfile.js}]
indent_style = space
indent_size = 2

[Makefile]
indent_style = tab
indent_size = 4


================================================
FILE: .gitignore
================================================
node_modules


================================================
FILE: .travis.yml
================================================
language: node_js
node_js:
- 4.2
- 4.1
- 0.12
- 0.11
- 0.10
notifications:
  email:
  - kimmo.brunfeldt+urltoimage@gmail.com


================================================
FILE: CHANGELOG.md
================================================
# Changelog

API changes across versions

## 0.2.0 -> 1.0.0

* `.done` callback is now: `.then`

    Promise library has been changed to [bluebird](http://bluebirdjs.com/docs/api-reference.html).

* `.fail` callback is now: `.catch`

* `options.ignoreSslErrors` was removed. Use `options.phantomArguments` instead.

    Examples
    ```
    {
        // Note: this is the new default for phantom arguments
        phantomArguments: '--ignore-ssl-errors=true'
    }
    ```

    ```
    urltoimage --phantom-arguments="--ignore-ssl-errors=true" google.com google.png
    ```

    See also: http://phantomjs.org/api/command-line.html

* `options.sslProtocol` was removed. Use `options.phantomArguments` instead.

    See above example.
    See also: http://phantomjs.org/api/command-line.html


================================================
FILE: LICENSE
================================================
The MIT License (MIT)

Copyright (c) 2015 Kimmo Brunfeldt

Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.


================================================
FILE: README.md
================================================
# url-to-image

**DEPRECATED:** I recommend using implementations that use Headless Chrome underneath, for example: https://github.com/kimmobrunfeldt/squint or https://github.com/alvarcarto/url-to-pdf-api. You can still continue using this module but no updates will be applied. 


[![Build Status](https://travis-ci.org/kimmobrunfeldt/url-to-image.png?branch=master)](https://travis-ci.org/kimmobrunfeldt/url-to-image)
[![Dependency Status](https://david-dm.org/kimmobrunfeldt/url-to-image.png?theme=shields.io)](https://david-dm.org/kimmobrunfeldt/url-to-image)
[![devDependency Status](https://david-dm.org/kimmobrunfeldt/url-to-image/dev-status.png?theme=shields.io)](https://david-dm.org/kimmobrunfeldt/url-to-image#info=devDependencies)

Takes screenshot of a given page. This module correctly handles pages which dynamically load content making AJAX requests.
Instead of waiting fixed amount of time before rendering, we give a short time for the page to make additional requests.

**Usage from code:**

```javascript
const urlToImage = require('url-to-image');
urlToImage('http://google.com', 'google.png').then(function() {
    // now google.png exists and contains screenshot of google.com
}).catch(function(err) {
    console.error(err);
});
```

**Usage from command line:**

```bash
$ urltoimage http://google.com google.png
```

Sometimes it's useful to see requests, responses and page errors from PhantomJS:

```bash
$ urltoimage http://google.com google.png --verbose
-> GET http://google.com/
-> GET http://www.google.fi/?gfe_rd=cr&ei=xTYxVouuOeiA8QexyZ2QBw
<- 302 http://google.com/
-> GET http://ssl.gstatic.com/gb/images/b_8d5afc09.png

... quite a lot of requests ...

-> GET http://ssl.gstatic.com/gb/js/sem_32d9c4210965b8e7bfa34fa376864ce8.js
<- 200 http://ssl.gstatic.com/gb/js/sem_32d9c4210965b8e7bfa34fa376864ce8.js
Render screenshot..
Done.
```


For more options, see [CLI](#command-line-interface-cli) chapter.

## Install

    npm install url-to-image

PhantomJS is installed by using [Medium/phantomjs NPM module](https://github.com/Medium/phantomjs).

## API

```javascript
const urlToImage = require('url-to-image');
```

#### urlToImage(url, filePath, options)

This will run a PhantomJS script([url-to-image.js](./src/url-to-image.js)) which renders given url to an image.

**Parameters**

* `url` Url of the page which will be rendered as image. For example `http://google.com`.
* `filePath` File path where to save rendered image.
* `options` Options for page rendering.

    **Default values for options**

    ```javascript
    {
        // User agent width
        width: 1200,

        // User agent height
        height: 800,
		
		// The file type of the rendered image. By default, PhantomJS 
		// sets the output format automatically based on the file extension.
		// Supported: PNG, GIF, JPEG, PDF
		fileType: 'jpeg',
		
		// The file quality of the rendered image, represented as a percentage. 
		// This reduces the image size. By default, 100 percent is used.
		fileQuality: 100,
		
		// Sets the width of the final image (cropped from the User agent defined size)
		// By default, no cropping is done.
		cropWidth: false,
		
		// Sets the height of the final image (cropped from the User agent defined size)
		// By default, no cropping is done.
		cropHeight: false,
		
		//Sets the offset of where to begin the image cropping from the left margin 
		// of the page
		cropOffsetLeft: 0,
	
    	//Sets the offset of where to begin the image cropping from the top margin 
		// of the page
		cropOffsetTop: 0,

        // How long in ms do we wait for additional requests
        // after all initial requests have gotten their response
        // Note: this does NOT limit the amount of time individual request
        //       can take in time
        requestTimeout: 300,

        // How long in ms do we wait at maximum. The screenshot is
        // taken after this time even though resources are not loaded
        maxTimeout: 1000 * 10,

        // How long in ms do we wait for phantomjs process to finish.
        // If the process is running after this time, it is killed.
        killTimeout: 1000 * 60 * 2,

        // If true, phantomjs script will output requests and responses to stdout
        verbose: false,

        // String of of phantomjs arguments
        // You can separate arguments with spaces
        // See options in http://phantomjs.org/api/command-line.html
        phantomArguments: '--ignore-ssl-errors=true'
    }
    ```

**Returns**

[Bluebird promise object](http://bluebirdjs.com/docs/api-reference.html).

**Detailed example**

```javascript
const urlToImage = require('url-to-image');

const options = {
    width: 600,
    height: 800,
    // Give a short time to load additional resources
    requestTimeout: 100
}

urlToImage('http://google.com', 'google.png', options)
.then(function() {
    // do stuff with google.png
})
.catch(function(err) {
    console.error(err);
});
```

## Command line interface (CLI)

The package also ships with a cli called `urltoimage`.

```
Usage: urltoimage <url> <path> [options]

<url>   Url to take screenshot of
<path>  File path where the screenshot is saved


Options:
  --width              Width of the viewport            [string] [default: 1280]
  --height             Height of the viewport            [string] [default: 800]
  --file-type          The file type of the rendered image. By default, 
		               PhantomJS sets the output format automatically based on 
					   the file extension. Supported: PNG, GIF, JPEG, PDF
					                                   [string] [default: false]
  --file-quality       The file quality of the rendered image, represented as a 
                       percentage. This reduces the image size. 
					   By default, 100 percent is used.
					                                     [string] [default: 100]
  --crop-width         Sets the width of the final image (cropped from the User 
                       agent defined size).
					                                   [string] [default: false]
  --crop-height        Sets the height of the final image (cropped from the User 
                       agent defined size).
					                                   [string] [default: false]
  --cropoffset-left    Sets the offset of where to begin the image cropping from 
                       the left margin of the page.
					                                   [string] [default: false]
  --cropoffset-top     Sets the offset of where to begin the image cropping from 
                       the top margin of the page.
					                                   [string] [default: false]
  --request-timeout    How long in ms do we wait for additional requests after
                       all initial requests have gotten their response
                                                         [string] [default: 300]
  --max-timeout        How long in ms do we wait at maximum. The screenshot is
                       taken after this time even though resources are not
                       loaded                          [string] [default: 10000]
  --kill-timeout       How long in ms do we wait for phantomjs process to
                       finish. If the process is running after this time, it is
                       killed.                        [string] [default: 120000]
  --phantom-arguments  Command line arguments to be passed to phantomjs
                       process.You must use the format
                       --phantom-arguments="--version".
                                  [string] [default: "--ignore-ssl-errors=true"]
  --verbose            If set, script will output additional information to
                       stdout.                        [boolean] [default: false]
  -h, --help           Show help                                       [boolean]
  -v, --version        Show version number                             [boolean]

Examples:
  urltoimage http://google.com google.png
```

# Contributors

## Release

* Commit all changes.
* Use [releasor](https://github.com/kimmobrunfeldt/releasor) to automate the release:

    `releasor --bump patch`

* Edit GitHub release notes.

## Test

    npm test

## Attribution

This module was inspired by

* [url-to-screenshot](https://github.com/juliangruber/url-to-screenshot)
* https://gist.github.com/cjoudrey/1341747

# License

MIT


================================================
FILE: package.json
================================================
{
  "name": "url-to-image",
  "version": "1.0.0",
  "description": "PhantomJS screenshotting done right",
  "main": "src/index.js",
  "bin": {
    "urltoimage": "./src/index.js"
  },
  "repository": {
    "type": "git",
    "url": "git@github.com:kimmobrunfeldt/url-to-image.git"
  },
  "keywords": [
    "phantomjs",
    "screenshot",
    "picture",
    "webpage",
    "render"
  ],
  "author": "Kimmo Brunfeldt",
  "license": "MIT",
  "bugs": {
    "url": "https://github.com/kimmobrunfeldt/url-to-image/issues"
  },
  "dependencies": {
    "bluebird": "^3.5.0",
    "lodash": "^4.17.4",
    "phantomjs-prebuilt": "^2.1.7",
    "yargs": "^8.0.2"
  },
  "devDependencies": {
    "image-size": "^0.5.5",
    "mocha": "^3.4.2",
    "releasor": "^1.2.1"
  },
  "scripts": {
    "test": "mocha"
  },
  "engines": {
    "node": ">=7.0"
  }
}


================================================
FILE: src/cli-parser.js
================================================
'use strict';

const _ = require('lodash');
const yargs = require('yargs');

const VERSION = require('../package.json').version;

const defaultOpts = {
    width: 1280,
    height: 800,
    requestTimeout: 300,
    maxTimeout: 1000 * 10,
    killTimeout: 1000 * 60 * 2,
    verbose: false,
    fileType: false,
    fileQuality: false,
    cropWidth: false,
    cropHeight: false,
    cropOffsetLeft: 0,
    cropOffsetTop: 0,
    phantomArguments: '--ignore-ssl-errors=true'
};

function getOpts(argv) {
    const userOpts = getUserOpts();
    const opts = _.merge(defaultOpts, userOpts);
    return validateAndTransformOpts(opts);
}

function getUserOpts() {
    const userOpts = yargs
    .usage(
        'Usage: $0 <url> <path> [options]\n\n' +
        '<url>   Url to take screenshot of\n' +
        '<path>  File path where the screenshot is saved\n'
    )
    .example('$0 http://google.com google.png')
    .demand(2)
    .option('width', {
        describe: 'Width of the viewport',
        default: defaultOpts.width,
        type: 'string'
    })
    .option('height', {
        describe: 'Height of the viewport',
        default: defaultOpts.height,
        type: 'string'
    })
    .option('request-timeout', {
        describe: 'How long in ms do we wait for additional requests' +
                  ' after all initial requests have gotten their response',
        default: defaultOpts.requestTimeout,
        type: 'string'
    })
    .option('max-timeout', {
        describe: 'How long in ms do we wait at maximum. The screenshot is' +
                  ' taken after this time even though resources are not loaded',
        default: defaultOpts.maxTimeout,
        type: 'string'
    })
    .option('file-type', {
        describe: 'Defines the file type you want to create.',
        default: defaultOpts.fileType,
        type: 'string'
    })
    .option('file-quality', {
        describe: 'Defines the quality of the file you want rendered' +
        'as a percentage. Default is 100.',
        default: defaultOpts.fileQuality,
        type: 'string'
    })
    .option('crop-width', {
        describe: 'The width of the final image which will be created',
        default: defaultOpts.cropWidth,
        type: 'string'
    })
    .option('crop-height', {
        describe: 'The height of the final image which will be created',
        default: defaultOpts.cropHeight,
        type: 'string'
    })
    .option('cropoffset-left', {
        describe: 'The position offset from the left of the screen from' +
        ' where to start the image crop',
        default: defaultOpts.cropOffsetLeft,
        type: 'string'
    })
    .option('cropoffset-top', {
        describe: 'The position offset from the top of the screen from' +
        ' where to start the image crop',
        default: defaultOpts.cropOffsetTop,
        type: 'string'
    })
    .option('kill-timeout', {
        describe: 'How long in ms do we wait for phantomjs process to finish.' +
        ' If the process is running after this time, it is killed.',
        default: defaultOpts.killTimeout,
        type: 'string'
    })
    .option('phantom-arguments', {
        describe: 'Command line arguments to be passed to phantomjs process.' +
                  'You must use the format --phantom-arguments="--version".',
        default: defaultOpts.phantomArguments,
        type: 'string'
    })
    .option('verbose', {
        describe: 'If set, script will output additional information to stdout.',
        default: defaultOpts.verbose,
        type: 'boolean'
    })
    .help('h')
    .alias('h', 'help')
    .alias('v', 'version')
    .version(VERSION)
    .argv;

    userOpts.url = userOpts._[0];
    userOpts.path = userOpts._[1];
    return userOpts;
}

function validateAndTransformOpts(opts) {
    if (opts.width) {
        validateNumber(opts.width, 'Incorrect argument, width is not a number');
    }

    if (opts.height) {
        validateNumber(opts.height, 'Incorrect argument, height is not a number');
    }

    if (opts.requestTimeout) {
        validateNumber(opts.requestTimeout, 'Incorrect argument, request timeout is not a number');
    }

    if (opts.maxTimeout) {
        validateNumber(opts.maxTimeout, 'Incorrect argument, max timeout is not a number');
    }

    if (opts.killTimeout) {
        validateNumber(opts.killTimeout, 'Incorrect argument, kill timeout is not a number');
    }

    if (opts.fileQuality) {
        validateNumber(opts.fileQuality, 'Incorrect argument, file quality is not a number');
    }

    if (opts.cropWidth) {
        validateNumber(opts.cropWidth, 'Incorrect argument, crop width is not a number');
    }

    if (opts.cropHeight) {
        validateNumber(opts.cropHeight, 'Incorrect argument, crop height is not a number');
    }

    if (opts.cropOffsetLeft) {
        validateNumber(opts.killTimeout, 'Incorrect argument, crop offset left is not a number');
    }

    if (opts.cropOffsetTop) {
        validateNumber(opts.killTimeout, 'Incorrect argument, crop offset top is not a number');
    }

    return opts;
}

function validateNumber(val, message) {
    const number = Number(val);
    if (!_.isNumber(number)) {
        const err = message;
        err.argumentError = true;
        throw err;
    }
}

module.exports = {
    defaultOpts: defaultOpts,
    getOpts: getOpts
};


================================================
FILE: src/index.js
================================================
#!/usr/bin/env node
'use strict';

const Promise = require('bluebird');
const _ = require('lodash');
const path = require('path');
const childProcess = require('child_process');
const phantomjs = require('phantomjs')
const cliParser = require('./cli-parser');

function render(url, filePath, opts) {
    opts = _.extend(cliParser.defaultOpts, opts);

    let args = [];
    if (_.isString(opts.phantomArguments)) {
        args = opts.phantomArguments.split(' ');
    }

    if (!_.startsWith(url, 'http') &&
        !_.startsWith(url, 'https') &&
        !_.startsWith(url, 'file')) {
        url = 'http://' + url;
    }

    args = args.concat([
        path.join(__dirname, 'url-to-image.js'),
        url,
        filePath,
        opts.width,
        opts.height,
        opts.requestTimeout,
        opts.maxTimeout,
        opts.verbose,
        opts.fileType,
        opts.fileQuality,
        opts.cropWidth,
        opts.cropHeight,
        opts.cropOffsetLeft,
        opts.cropOffsetTop
    ]);

    let execOpts = {
        maxBuffer: Infinity
    };

    let killTimer;
    return new Promise(function(resolve, reject) {
        let child;
        killTimer = setTimeout(function() {
            killPhantom(opts, child);
            reject(new Error('Phantomjs process timeout'));
        }, opts.killTimeout);

        try {
            child = childProcess.spawn(phantomjs.path, args, {
                stdio: 'inherit'
            });
        } catch (err) {
            return Promise.reject(err);
        }

        function errorHandler(err) {
            // Remove bound handlers after use
            child.removeListener('close', closeHandler);
            reject(err);
        }

        function closeHandler(exitCode) {
            child.removeListener('error', errorHandler);
            if (exitCode > 0) {
                let err;
                if (exitCode === 10) {
                    err = new Error('Unable to load given url: ' + url);
                }
                reject(err);
            } else {
                resolve(exitCode);
            }
        }

        child.once('error', errorHandler);
        child.once('close', closeHandler);
    })
    .finally(function() {
        if (killTimer) {
            clearTimeout(killTimer);
        }
    });
}

function killPhantom(opts, child) {
    if (child) {
        const msg = 'Phantomjs process didn\'t finish in ' +
                  opts.killTimeout + 'ms, killing it..';
        console.error(msg);

        child.kill();
    }
}

if (require.main === module) {
    let opts;
    try {
        opts = cliParser.getOpts();
    } catch (err) {
        if (err.argumentError) {
            console.error(err.message);
            process.exit(1);
        }

        throw err;
    }

    render(opts.url, opts.path, opts)
    .catch(function(err) {
        console.error('\nTaking screenshot failed to error:');
        if (err && err.message) {
            console.error(err.message);
        } else if (err) {
            console.error(err);
        } else {
            console.error('No error message available');
        }

        process.exit(2);
    });
}

module.exports = render;


================================================
FILE: src/url-to-image.js
================================================
'use strict';

// PhantomJS script
// Takes screeshot of a given page. This correctly handles pages which
// dynamically load content making AJAX requests.

// Instead of waiting fixed amount of time before rendering, we give a short
// time for the page to make additional requests.

// Phantom internals
const system = require('system');
const webPage = require('webpage');

function main() {
    // I tried to use yargs as a nicer commandline option parser but
    // it doesn't run in phantomjs environment
    const args = system.args;
    const opts = {
        url: args[1],
        filePath: args[2],
        width: args[3],
        height: args[4],
        requestTimeout: args[5],
        maxTimeout: args[6],
        verbose: args[7] === 'true',
        fileType: args[8],
        fileQuality: args[9] ? args[9] : 100,
        cropWidth: args[10],
        cropHeight: args[11],
        cropOffsetLeft: args[12] ? args[12] : 0,
        cropOffsetTop: args[13] ? args[13] : 0
    };

    renderPage(opts);
}

function renderPage(opts) {
    let requestCount = 0;
    let forceRenderTimeout;
    let dynamicRenderTimeout;

    const page = webPage.create();
    page.viewportSize = {
        width: opts.width,
        height: opts.height
    };
    // Silence confirmation messages and errors
    page.onConfirm = page.onPrompt = function noOp() {};
    page.onError = function(err) {
        log('Page error:', err);
    };

    page.onResourceRequested = function(request) {
        log('->', request.method, request.url);
        requestCount += 1;
        clearTimeout(dynamicRenderTimeout);
    };

    page.onResourceReceived = function(response) {
        if (!response.stage || response.stage === 'end') {
            log('<-', response.status, response.url);
            requestCount -= 1;
            if (requestCount === 0) {
                dynamicRenderTimeout = setTimeout(renderAndExit, opts.requestTimeout);
            }
        }
    };

    page.open(opts.url, function(status) {
        if (status !== 'success') {
            log('Unable to load url:', opts.url);
            phantom.exit(10);
        } else {
            forceRenderTimeout = setTimeout(renderAndExit, opts.maxTimeout);
        }
    });

    function log() {
        // PhantomJS doesn't stringify objects very well, doing that manually
        if (opts.verbose) {
            let args = Array.prototype.slice.call(arguments);

            let str = '';
            args.forEach(function(arg) {
                if (isString) {
                    str += arg;
                } else {
                    str += JSON.stringify(arg, null, 2);
                }

                str += ' '
            });

            console.log(str);
        }
    }

    function renderAndExit() {
        log('Render screenshot..');
        if(opts.cropWidth && opts.cropHeight) {
        log("Cropping...");
            page.clipRect = {top: opts.cropOffsetTop, left: opts.cropOffsetLeft, width: opts.cropWidth, height: opts.cropHeight};
        }

        let renderOpts = {
            fileQuality: opts.fileQuality
        };

        if(opts.fileType) {
            log("Adjusting File Type...");
            renderOpts.fileType = opts.fileType;
        }

        page.render(opts.filePath, renderOpts);
        log('Done.');
        phantom.exit();
    }
}

function isString(value) {
    return typeof value === 'string'
}

main();


================================================
FILE: test/test-functional.js
================================================
'use strict';

const http = require('http');
const fs = require('fs');
const assert = require('assert');
const sizeOf = require('image-size');

const urlToImage = require('../src/index');

describe('urlToImage', function() {

    const server = http.createServer(function(req, res) {
        res.end('<html>test</html>');
    });

    before(function(done) {
        server.listen(9000);
        server.on('listening', done);
    });

    after(function(done) {
        server.close();
        done();
    });

    describe('render', function() {
        this.timeout(20000);

        it('should render test image', function(done) {
            urlToImage('http://localhost:9000', 'localhost.png')
            .then(function() {
                const dimensions = sizeOf('localhost.png');
                assert.equal(dimensions.width, 1280, 'default width is incorrect');
                fs.unlinkSync('localhost.png');
                done();
            });
        });

        it('should render image in custom size', function(done) {
            urlToImage(
                'http://localhost:9000',
                'localhost.png', {
                    width: 800,
                    height: 600
                }
            )
            .then(function() {
                const dimensions = sizeOf('localhost.png');

                assert.equal(dimensions.width, 800, 'width is incorrect');

                // The content of test page is so small, so viewport
                // is larger than content. If content were larger,
                // urlToImage's height could be bigger than viewport's width
                assert.equal(dimensions.height, 600, 'height is incorrect');

                fs.unlinkSync('localhost.png');
                done();
            });
        });

        it('should fail to incorrect url', function(done) {
            this.timeout(5000);

            urlToImage(
                'http://failure',
                'localhost.png', {
                    width: 800,
                    height: 600
                }
            )
            .catch(function(err) {
                done();
            });
        });
    });
});